vm-test-run-nixbot-gitlab
checks.x86_64-linux.nixbot-gitlab
· build #351
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab: QEMU running (pid 46)10gitlab # Disk image does not exist, creating the virtualisation disk image...11(finished: start all VMs, in 47.37 seconds)12gitlab # Formatting '/build/vm-state-gitlab/tmp.klSOEaq9md', fmt=raw size=107374182413gitlab: waiting for unit setup-gitlab.service14gitlab # mke2fs 1.47.4 (6-Mar-2025)15gitlab # Discarding device blocks: 0/262144 done16gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes17gitlab: waiting for the VM to finish booting18gitlab # Filesystem UUID: ef5f3dc4-b9b2-4c64-b653-4134e059cbfb19gitlab # Superblock backups stored on blocks:20gitlab # 32768, 98304, 163840, 22937621gitlab # 22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)31gitlab # 32gitlab # 33gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA0034gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...35gitlab # 36gitlab # 37gitlab # 38gitlab # 39gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB0040gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...41gitlab # 42gitlab # 43gitlab # Booting from ROM...44gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 202645gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/wlx59kvgfj7m7h8yjhv348h405rq3xx9-nixos-system-gitlab-test/init regInfo=/nix/store/4a7194bll89l9cc0jq8fb9fmga38fbga-closure-info/registration console=ttyS0,115200n8 console=tty046gitlab # [ 0.000000] BIOS-provided physical RAM map:47gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable48gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved49gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved50gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable51gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved52gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved53gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved54gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable55gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved56gitlab # [ 0.000000] NX (Execute Disable) protection: active57gitlab # [ 0.000000] APIC: Static calls initialized58gitlab # [ 0.000000] SMBIOS 2.8 present.59gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/201460gitlab # [ 0.000000] DMI: Memory slots populated: 1/161gitlab # [ 0.000000] Hypervisor detected: KVM62gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000063gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d0064gitlab # [ 0.000000] kvm-clock: using sched offset of 477667334 cycles65gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns66gitlab # [ 0.000005] tsc: Detected 2400.012 MHz processor67gitlab # [ 0.000810] last_pfn = 0x1c0000 max_arch_pfn = 0x1000000000068gitlab # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs69gitlab # [ 0.000839] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT70gitlab # [ 0.000877] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000071gitlab # [ 0.002732] found SMP MP-table at [mem 0x000f5470-0x000f547f]72gitlab # [ 0.002744] Using GB pages for direct mapping73gitlab # [ 0.002870] RAMDISK: [mem 0xbe31c000-0xbffcffff]74gitlab # [ 0.002876] ACPI: Early table checksum verification disabled75gitlab # [ 0.002879] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )76gitlab # [ 0.002883] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)77gitlab # [ 0.002886] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)78gitlab # [ 0.002893] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001)79gitlab # [ 0.002895] ACPI: FACS 0x00000000BFFE0000 00004080gitlab # [ 0.002896] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001)81gitlab # [ 0.002898] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)82gitlab # [ 0.002899] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)83gitlab # [ 0.002901] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]84gitlab # [ 0.002901] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]85gitlab # [ 0.002902] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]86gitlab # [ 0.002902] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]87gitlab # [ 0.002903] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]88gitlab # [ 0.002903] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]89gitlab # [ 0.003121] No NUMA configuration found90gitlab # [ 0.003122] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]91gitlab # [ 0.003125] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff]92gitlab # [ 0.005340] Zone ranges:93gitlab # [ 0.005341] DMA [mem 0x0000000000001000-0x0000000000ffffff]94gitlab # [ 0.005342] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]95gitlab # [ 0.005344] Normal [mem 0x0000000100000000-0x00000001bfffffff]96gitlab # [ 0.005345] Device empty97gitlab # [ 0.005346] Movable zone start for each node98gitlab # [ 0.005346] Early memory node ranges99gitlab # [ 0.005347] node 0: [mem 0x0000000000001000-0x000000000009efff]100gitlab # [ 0.005348] node 0: [mem 0x0000000000100000-0x00000000bffd7fff]101gitlab # [ 0.005349] node 0: [mem 0x0000000100000000-0x00000001bfffffff]102gitlab # [ 0.005350] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]103gitlab # [ 0.005369] On node 0, zone DMA: 1 pages in unavailable ranges104gitlab # [ 0.005633] On node 0, zone DMA: 97 pages in unavailable ranges105gitlab # [ 0.109460] On node 0, zone Normal: 40 pages in unavailable ranges106gitlab # [ 0.109893] ACPI: PM-Timer IO Port: 0x608107gitlab # [ 0.109909] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])108gitlab # [ 0.109946] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23109gitlab # [ 0.109949] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)110gitlab # [ 0.109951] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)111gitlab # [ 0.109952] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)112gitlab # [ 0.109953] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)113gitlab # [ 0.109954] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)114gitlab # [ 0.109956] ACPI: Using ACPI (MADT) for SMP configuration information115gitlab # [ 0.109957] ACPI: HPET id: 0x8086a201 base: 0xfed00000116gitlab # [ 0.109963] TSC deadline timer available117gitlab # [ 0.109967] CPU topo: Max. logical packages: 1118gitlab # [ 0.109968] CPU topo: Max. logical dies: 1119gitlab # [ 0.109968] CPU topo: Max. dies per package: 1120gitlab # [ 0.109972] CPU topo: Max. threads per core: 1121gitlab # [ 0.109972] CPU topo: Num. cores per package: 4122gitlab # [ 0.109973] CPU topo: Num. threads per package: 4123gitlab # [ 0.109973] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs124gitlab # [ 0.109998] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()125gitlab # [ 0.110015] kvm-guest: KVM setup pv remote TLB flush126gitlab # [ 0.110018] kvm-guest: setup PV sched yield127gitlab # [ 0.110034] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]128gitlab # [ 0.110036] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]129gitlab # [ 0.110037] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]130gitlab # [ 0.110038] [mem 0xc0000000-0xfeffbfff] available for PCI devices131gitlab # [ 0.110040] Booting paravirtualized kernel on KVM132gitlab # [ 0.110044] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns133gitlab # [ 0.114488] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1134gitlab # [ 0.116638] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288135gitlab # [ 0.116692] kvm-guest: PV spinlocks enabled136gitlab # [ 0.116694] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)137gitlab # [ 0.116697] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/wlx59kvgfj7m7h8yjhv348h405rq3xx9-nixos-system-gitlab-test/init regInfo=/nix/store/4a7194bll89l9cc0jq8fb9fmga38fbga-closure-info/registration console=ttyS0,115200n8 console=tty0138gitlab # [ 0.116796] Unknown kernel command line parameters "regInfo=/nix/store/4a7194bll89l9cc0jq8fb9fmga38fbga-closure-info/registration", will be passed to user space.139gitlab # [ 0.116809] random: crng init done140gitlab # [ 0.116809] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes141gitlab # [ 0.125252] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)142gitlab # [ 0.129559] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)143gitlab # [ 0.129637] software IO TLB: area num 4.144gitlab # [ 0.200825] Fallback order for Node 0: 0145gitlab # [ 0.200834] Built 1 zonelists, mobility grouping on. Total pages: 1572726146gitlab # [ 0.200836] Policy zone: Normal147gitlab # [ 0.203286] mem auto-init: stack:all(zero), heap alloc:on, heap free:off148gitlab # [ 0.212898] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1149gitlab # [ 0.226019] allocated 12582912 bytes of page_ext150gitlab # [ 0.235893] ftrace: allocating 48717 entries in 192 pages151gitlab # [ 0.235896] ftrace: allocated 192 pages with 2 groups152gitlab # [ 0.236721] Dynamic Preempt: lazy153gitlab # [ 0.236895] rcu: Preemptible hierarchical RCU implementation.154gitlab # [ 0.236895] rcu: RCU event tracing is enabled.155gitlab # [ 0.236896] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.156gitlab # [ 0.236898] Trampoline variant of Tasks RCU enabled.157gitlab # [ 0.236898] Rude variant of Tasks RCU enabled.158gitlab # [ 0.236898] Tracing variant of Tasks RCU enabled.159gitlab # [ 0.236899] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.160gitlab # [ 0.236900] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4161gitlab # [ 0.236926] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.162gitlab # [ 0.236928] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.163gitlab # [ 0.236929] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.164gitlab # [ 0.241302] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16165gitlab # [ 0.241587] rcu: srcu_init: Setting srcu_struct sizes based on contention.166gitlab # [ 0.241596] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns167gitlab # [ 0.241708] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)168gitlab # [ 0.245328] Console: colour VGA+ 80x25169gitlab # [ 0.245331] printk: legacy console [tty0] enabled170gitlab # [ 0.275832] printk: legacy console [ttyS0] enabled171gitlab # [ 0.383534] ACPI: Core revision 20250807172gitlab # [ 0.384500] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns173gitlab # [ 0.386093] APIC: Switch to symmetric I/O mode setup174gitlab # [ 0.387107] x2apic enabled175gitlab # [ 0.387867] APIC: Switched APIC routing to: physical x2apic176gitlab # [ 0.388823] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()177gitlab # [ 0.390093] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()178gitlab # [ 0.391583] kvm-guest: setup PV IPIs179gitlab # [ 0.393163] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1180gitlab # [ 0.394200] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns181gitlab # [ 0.395929] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)182gitlab # [ 0.398019] x86/cpu: User Mode Instruction Prevention (UMIP) activated183gitlab # [ 0.399048] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127184gitlab # [ 0.400926] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0185gitlab # [ 0.401930] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto186gitlab # [ 0.402926] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl187gitlab # [ 0.404926] Transient Scheduler Attacks: Vulnerable: No microcode188gitlab # [ 0.405926] Spectre V2 : Mitigation: Enhanced / Automatic IBRS189gitlab # [ 0.406926] Speculative Return Stack Overflow: Mitigation: Safe RET190gitlab # [ 0.407926] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization191gitlab # [ 0.408934] Spectre V2 : Enabling IBPB for BPF192gitlab # [ 0.410506] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier193gitlab # [ 0.411926] active return thunk: srso_alias_return_thunk194gitlab # [ 0.412947] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'195gitlab # [ 0.413926] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'196gitlab # [ 0.414926] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'197gitlab # [ 0.415926] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'198gitlab # [ 0.416926] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'199gitlab # [ 0.417926] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'200gitlab # [ 0.418926] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'201gitlab # [ 0.420926] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256202gitlab # [ 0.421926] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64203gitlab # [ 0.422926] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512204gitlab # [ 0.423926] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024205gitlab # [ 0.424926] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8206gitlab # [ 0.425926] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.207gitlab # [ 0.455813] Freeing SMP alternatives memory: 44K208gitlab # [ 0.456524] pid_max: default: 32768 minimum: 301209gitlab # [ 0.457042] LSM: initializing lsm=capability,landlock,yama,bpf,ima210gitlab # [ 0.458029] landlock: Up and running.211gitlab # [ 0.458926] Yama: becoming mindful.212gitlab # [ 0.460135] LSM support for eBPF active213gitlab # [ 0.461136] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)214gitlab # [ 0.462073] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)215gitlab # [ 0.465024] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)216gitlab # [ 0.466500] Performance Events: Fam17h+ core perfctr, AMD PMU driver.217gitlab # [ 0.466935] ... version: 2218gitlab # [ 0.467685] ... bit width: 48219gitlab # [ 0.467928] ... generic counters: 6220gitlab # [ 0.468674] ... generic bitmap: 000000000000003f221gitlab # [ 0.468929] ... fixed-purpose counters: 0222gitlab # [ 0.469676] ... fixed-purpose bitmap: 0000000000000000223gitlab # [ 0.469929] ... value mask: 0000ffffffffffff224gitlab # [ 0.470847] ... max period: 00007fffffffffff225gitlab # [ 0.471660] ... global_ctrl mask: 000000000000003f226gitlab # [ 0.472030] signal: max sigframe size: 3376227gitlab # [ 0.472823] rcu: Hierarchical SRCU implementation.228gitlab # [ 0.473553] rcu: Max phase no-delay instances is 400.229gitlab # [ 0.474108] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level230gitlab # [ 0.479481] smp: Bringing up secondary CPUs ...231gitlab # [ 0.480561] smpboot: x86: Booting SMP configuration:232gitlab # [ 0.480931] .... node #0, CPUs: #1 #2 #3233gitlab # [ 0.484072] smp: Brought up 1 node, 4 CPUs234gitlab # [ 0.485645] smpboot: Total of 4 processors activated (19200.09 BogoMIPS)235gitlab # [ 0.486426] Memory: 6011548K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260108K reserved, 0K cma-reserved)236gitlab # [ 0.487929] devtmpfs: initialized237gitlab # [ 0.489114] x86/mm: Memory block size: 128MB238gitlab # [ 0.492972] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)239gitlab # [ 0.494007] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).240gitlab # [ 0.495031] pinctrl core: initialized pinctrl subsystem241gitlab # [ 0.496244] PM: RTC time: 16:26:11, date: 2026-09-06242gitlab # [ 0.499906] NET: Registered PF_NETLINK/PF_ROUTE protocol family243gitlab # [ 0.502033] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations244gitlab # [ 0.502990] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations245gitlab # [ 0.504987] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations246gitlab # [ 0.505940] audit: initializing netlink subsys (disabled)247gitlab # [ 0.506989] audit: type=2000 audit(1788711972.804:1): state=initialized audit_enabled=0 res=1248gitlab # [ 0.507253] thermal_sys: Registered thermal governor 'fair_share'249gitlab # [ 0.508929] thermal_sys: Registered thermal governor 'bang_bang'250gitlab # [ 0.509928] thermal_sys: Registered thermal governor 'step_wise'251gitlab # [ 0.510928] thermal_sys: Registered thermal governor 'user_space'252gitlab # [ 0.511918] thermal_sys: Registered thermal governor 'power_allocator'253gitlab # [ 0.512747] cpuidle: using governor menu254gitlab # [ 0.515142] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5255gitlab # [ 0.516142] PCI: Using configuration type 1 for base access256gitlab # [ 0.516929] PCI: Using configuration type 1 for extended access257gitlab # [ 0.518095] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.258gitlab # [ 0.520986] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages259gitlab # [ 0.521929] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page260gitlab # [ 0.522931] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages261gitlab # [ 0.523929] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page262gitlab # [ 0.528217] ACPI: Added _OSI(Module Device)263gitlab # [ 0.528930] ACPI: Added _OSI(Processor Device)264gitlab # [ 0.529722] ACPI: Added _OSI(Processor Aggregator Device)265gitlab # [ 0.531896] ACPI: 1 ACPI AML tables successfully acquired and loaded266gitlab # [ 0.535939] ACPI: Interpreter enabled267gitlab # [ 0.537014] ACPI: PM: (supports S0 S3 S4 S5)268gitlab # [ 0.538942] ACPI: Using IOAPIC for interrupt routing269gitlab # [ 0.541028] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug270gitlab # [ 0.544937] PCI: Using E820 reservations for host bridge windows271gitlab # [ 0.547365] ACPI: Enabled 2 GPEs in block 00 to 0F272gitlab # [ 0.561248] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])273gitlab # [ 0.562947] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]274gitlab # [ 0.567940] acpiphp: Slot [3] registered275gitlab # [ 0.569987] acpiphp: Slot [4] registered276gitlab # [ 0.571004] acpiphp: Slot [5] registered277gitlab # [ 0.573049] acpiphp: Slot [6] registered278gitlab # [ 0.574988] acpiphp: Slot [7] registered279gitlab # [ 0.576983] acpiphp: Slot [8] registered280gitlab # [ 0.578019] acpiphp: Slot [9] registered281gitlab # [ 0.579986] acpiphp: Slot [10] registered282gitlab # [ 0.581985] acpiphp: Slot [11] registered283gitlab # [ 0.582985] acpiphp: Slot [12] registered284gitlab # [ 0.584990] acpiphp: Slot [13] registered285gitlab # [ 0.585971] acpiphp: Slot [14] registered286gitlab # [ 0.586970] acpiphp: Slot [15] registered287gitlab # [ 0.588989] acpiphp: Slot [16] registered288gitlab # [ 0.589971] acpiphp: Slot [17] registered289gitlab # [ 0.590971] acpiphp: Slot [18] registered290gitlab # [ 0.592970] acpiphp: Slot [19] registered291gitlab # [ 0.593997] acpiphp: Slot [20] registered292gitlab # [ 0.594971] acpiphp: Slot [21] registered293gitlab # [ 0.596972] acpiphp: Slot [22] registered294gitlab # [ 0.597980] acpiphp: Slot [23] registered295gitlab # [ 0.600492] acpiphp: Slot [24] registered296gitlab # [ 0.602009] acpiphp: Slot [25] registered297gitlab # [ 0.603991] acpiphp: Slot [26] registered298gitlab # [ 0.605011] acpiphp: Slot [27] registered299gitlab # [ 0.606987] acpiphp: Slot [28] registered300gitlab # [ 0.608997] acpiphp: Slot [29] registered301gitlab # [ 0.610988] acpiphp: Slot [30] registered302gitlab # [ 0.612011] acpiphp: Slot [31] registered303gitlab # [ 0.614050] PCI host bridge to bus 0000:00304gitlab # [ 0.615946] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]305gitlab # [ 0.618935] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]306gitlab # [ 0.620936] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]307gitlab # [ 0.623936] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]308gitlab # [ 0.626935] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]309gitlab # [ 0.630937] pci_bus 0000:00: root bus resource [bus 00-ff]310gitlab # [ 0.633626] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint311gitlab # [ 0.638247] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint312gitlab # [ 0.642000] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint313gitlab # [ 0.648988] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f]314gitlab # [ 0.650987] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk315gitlab # [ 0.652932] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk316gitlab # [ 0.654933] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk317gitlab # [ 0.656933] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk318gitlab # [ 0.659300] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint319gitlab # [ 0.664017] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f]320gitlab # [ 0.666424] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint321gitlab # [ 0.668217] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI322gitlab # [ 0.670953] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB323gitlab # [ 0.674524] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint324gitlab # [ 0.700060] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]325gitlab # [ 0.701960] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]326gitlab # [ 0.703978] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]327gitlab # [ 0.706732] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]328gitlab # [ 0.710320] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint329gitlab # [ 0.718947] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]330gitlab # [ 0.721944] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]331gitlab # [ 0.722974] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]332gitlab # [ 0.736980] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]333gitlab # [ 0.739743] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint334gitlab # [ 0.744940] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f]335gitlab # [ 0.745829] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]336gitlab # [ 0.746720] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]337gitlab # [ 0.752363] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint338gitlab # [ 0.760957] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf]339gitlab # [ 0.763951] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]340gitlab # [ 0.765988] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]341gitlab # [ 0.771530] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint342gitlab # [ 0.778947] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df]343gitlab # [ 0.780945] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]344gitlab # [ 0.782981] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]345gitlab # [ 0.787851] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint346gitlab # [ 0.792949] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]347gitlab # [ 0.793859] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]348gitlab # [ 0.794717] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]349gitlab # [ 0.797643] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint350gitlab # [ 0.800937] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff]351gitlab # [ 0.801828] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]352gitlab # [ 0.803709] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]353gitlab # [ 0.806088] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint354gitlab # [ 0.814958] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff]355gitlab # [ 0.816952] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]356gitlab # [ 0.819989] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]357gitlab # [ 0.822948] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]358gitlab # [ 0.828775] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint359gitlab # [ 0.836969] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]360gitlab # [ 0.839004] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]361gitlab # [ 0.845389] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint362gitlab # [ 0.854124] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f]363gitlab # [ 0.856945] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]364gitlab # [ 0.857975] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]365gitlab # [ 0.863368] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint366gitlab # [ 0.872972] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f]367gitlab # [ 0.873939] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]368gitlab # [ 0.874876] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]369gitlab # [ 0.880381] ACPI: PCI: Interrupt link LNKA configured for IRQ 10370gitlab # [ 0.881083] ACPI: PCI: Interrupt link LNKB configured for IRQ 10371gitlab # [ 0.882063] ACPI: PCI: Interrupt link LNKC configured for IRQ 11372gitlab # [ 0.884114] ACPI: PCI: Interrupt link LNKD configured for IRQ 11373gitlab # [ 0.885007] ACPI: PCI: Interrupt link LNKS configured for IRQ 9374gitlab # [ 0.887942] iommu: Default domain type: Translated375gitlab # [ 0.888784] iommu: DMA domain TLB invalidation policy: lazy mode376gitlab # [ 0.890250] ACPI: bus type USB registered377gitlab # [ 0.891067] usbcore: registered new interface driver usbfs378gitlab # [ 0.891992] usbcore: registered new interface driver hub379gitlab # [ 0.892899] usbcore: registered new device driver usb380gitlab # [ 0.894873] NetLabel: Initializing381gitlab # [ 0.895387] NetLabel: domain hash size = 128382gitlab # [ 0.896929] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO383gitlab # [ 0.897974] NetLabel: unlabeled traffic allowed by default384gitlab # [ 0.898907] PCI: Using ACPI for IRQ routing385gitlab # [ 0.900086] pci 0000:00:02.0: vgaarb: setting as boot VGA device386gitlab # [ 0.900925] pci 0000:00:02.0: vgaarb: bridge control possible387gitlab # [ 0.900925] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none388gitlab # [ 0.900932] vgaarb: loaded389gitlab # [ 0.901726] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0390gitlab # [ 0.902569] hpet0: 3 comparators, 64-bit 100.000000 MHz counter391gitlab # [ 0.907025] clocksource: Switched to clocksource kvm-clock392gitlab # [ 0.908675] VFS: Disk quotas dquot_6.6.0393gitlab # [ 0.909421] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)394gitlab # [ 0.910942] pnp: PnP ACPI init395gitlab # [ 0.912092] pnp: PnP ACPI: found 6 devices396gitlab # [ 0.921706] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns397gitlab # [ 0.923278] clocksource: Switched to clocksource acpi_pm398gitlab # [ 0.924313] NET: Registered PF_INET protocol family399gitlab # [ 0.926433] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)400gitlab # [ 0.943727] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)401gitlab # [ 0.945226] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)402gitlab # [ 0.946647] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)403gitlab # [ 0.950285] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)404gitlab # [ 0.951659] TCP: Hash tables configured (established 65536 bind 65536)405gitlab # [ 0.953111] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)406gitlab # [ 0.954714] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)407gitlab # [ 0.956307] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)408gitlab # [ 0.957696] NET: Registered PF_UNIX/PF_LOCAL protocol family409gitlab # [ 0.958680] NET: Registered PF_XDP protocol family410gitlab # [ 0.959553] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]411gitlab # [ 0.960598] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]412gitlab # [ 0.961642] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]413gitlab # [ 0.962786] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]414gitlab # [ 0.963927] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]415gitlab # [ 0.965275] pci 0000:00:01.0: PIIX3: Enabling Passive Release416gitlab # [ 0.966281] pci 0000:00:00.0: Limiting direct PCI/PCI transfers417gitlab # [ 0.968730] ACPI: \_SB_.LNKD: Enabled at IRQ 11418gitlab # [ 0.971261] PCI: CLS 0 bytes, default 64419gitlab # [ 0.972069] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)420gitlab # [ 0.972752] Trying to unpack rootfs image as initramfs...421gitlab # [ 0.973142] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB)422gitlab # [ 0.975938] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns423gitlab # [ 0.996449] Initialise system trusted keyrings424gitlab # [ 0.997751] workingset: timestamp_bits=40 max_order=21 bucket_order=0425gitlab # [ 1.009288] Key type asymmetric registered426gitlab # [ 1.010040] Asymmetric key parser 'x509' registered427gitlab # [ 1.010985] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)428gitlab # [ 1.012556] io scheduler mq-deadline registered429gitlab # [ 1.013371] io scheduler kyber registered430gitlab # [ 1.016810] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled431gitlab # [ 1.018174] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A432gitlab # [ 1.020806] Linux agpgart interface v0.103433gitlab # [ 1.021619] ACPI: bus type drm_connector registered434gitlab # [ 1.024769] usbcore: registered new interface driver usbserial_generic435gitlab # [ 1.025891] usbserial: USB Serial support registered for generic436gitlab # [ 1.026913] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.437gitlab # [ 1.026913] Please enable it if your BIOS has the CPPC option.438gitlab # [ 1.029252] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled439gitlab # [ 1.030693] drop_monitor: Initializing network drop monitor service440gitlab # [ 1.031921] NET: Registered PF_INET6 protocol family441gitlab # [ 1.033799] Segment Routing with IPv6442gitlab # [ 1.034500] In-situ OAM (IOAM) with IPv6443gitlab # [ 1.036660] IPI shorthand broadcast: enabled444gitlab # [ 1.041084] sched_clock: Marking stable (894010676, 146329158)->(1064206629, -23866795)445gitlab # [ 1.042931] registered taskstats version 1446gitlab # [ 1.043984] Loading compiled-in X.509 certificates447gitlab # [ 1.053967] Demotion targets for Node 0: null448gitlab # [ 1.055074] Key type .fscrypt registered449gitlab # [ 1.055805] Key type fscrypt-provisioning registered450gitlab # [ 1.056772] ima: No TPM chip found, activating TPM-bypass!451gitlab # [ 1.057727] ima: Allocated hash algorithm: sha1452gitlab # [ 1.058560] ima: No architecture policies found453gitlab # [ 1.059662] PM: Magic number: 10:540:438454gitlab # [ 1.061265] RAS: Correctable Errors collector initialized.455gitlab # [ 1.065740] clk: Disabling unused clocks456gitlab # [ 1.066472] PM: genpd: Disabling unused power domains457gitlab # [ 1.258639] Freeing initrd memory: 29392K458gitlab # [ 1.265688] Freeing unused decrypted memory: 2028K459gitlab # [ 1.271904] Freeing unused kernel image (initmem) memory: 3640K460gitlab # [ 1.274612] Write protecting the kernel read-only data: 32768k461gitlab # [ 1.279389] Freeing unused kernel image (text/rodata gap) memory: 1220K462gitlab # [ 1.283130] Freeing unused kernel image (rodata/data gap) memory: 760K463gitlab # [ 1.351899] x86/mm: Checked W+X mappings: passed, no W+X pages found.464gitlab # [ 1.353067] Run /init as init process465gitlab # [ 1.365890] systemd[1]: Inserted module 'autofs4'466gitlab # [ 1.390876] fuse: init (API version 7.45)467gitlab # [ 1.399456] ACPI: \_SB_.LNKC: Enabled at IRQ 10468gitlab # [ 1.406680] ACPI: \_SB_.LNKA: Enabled at IRQ 10469gitlab # [ 1.409792] ACPI: \_SB_.LNKB: Enabled at IRQ 11470gitlab # [ 1.449400] systemd[1]: Successfully made /usr/ read-only.471gitlab # [ 1.786289] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)472gitlab # [ 1.826142] systemd[1]: Detected virtualization kvm.473gitlab # [ 1.828445] systemd[1]: Detected architecture x86-64.474gitlab # [ 1.830596] systemd[1]: Running in initrd.475gitlab # [ 1.833052] systemd[1]: Initializing machine ID from random generator.476gitlab # [ 1.836080] systemd[1]: Hostname set to <gitlab>.477gitlab # [ 1.951662] systemd[1]: bpf-restrict-fs: LSM BPF program attached478gitlab # [ 1.995686] systemd[1]: Queued start job for default target Initrd Default Target.479gitlab # [ 2.007937] systemd[1]: Created slice Slice /system/modprobe.480gitlab # [ 2.009147] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.481gitlab # [ 2.010625] systemd[1]: Expecting device /dev/disk/by-label/nix-store...482gitlab # [ 2.011812] systemd[1]: Expecting device /dev/disk/by-label/nixos...483gitlab # [ 2.012954] systemd[1]: Reached target Path Units.484gitlab # [ 2.013871] systemd[1]: Reached target Slice Units.485gitlab # [ 2.014801] systemd[1]: Reached target Swaps.486gitlab # [ 2.015642] systemd[1]: Reached target Timer Units.487gitlab # [ 2.016664] systemd[1]: Listening on D-Bus System Message Bus Socket.488gitlab # [ 2.017904] systemd[1]: Listening on Journal Socket (/dev/log).489gitlab # [ 2.019090] systemd[1]: Listening on Journal Sockets.490gitlab # [ 2.020089] systemd[1]: Listening on udev Control Socket.491gitlab # [ 2.021098] systemd[1]: Listening on udev Kernel Socket.492gitlab # [ 2.022055] systemd[1]: Reached target Socket Units.493gitlab # [ 2.024158] systemd[1]: Starting Create List of Static Device Nodes...494gitlab # [ 2.026253] systemd[1]: Starting Load Kernel Module 9pnet_virtio...495gitlab # [ 2.028392] systemd[1]: Starting Load Kernel Module configfs...496gitlab # [ 2.032104] systemd[1]: Starting Journal Service...497gitlab # [ 2.043451] systemd[1]: Starting Load Kernel Modules...498gitlab # [ 2.044516] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os499gitlab # [ 2.053520] systemd[1]: Starting Coldplug All udev Devices...500gitlab # [ 2.053705] systemd-journald[90]: Collecting audit messages is disabled.501gitlab # [ 2.061763] systemd[1]: Finished Create List of Static Device Nodes.502gitlab # [ 2.066935] systemd[1]: modprobe@configfs.service: Deactivated successfully.503gitlab # [ 2.080202] systemd[1]: Finished Load Kernel Module configfs.504gitlab # [ 2.091997] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config505gitlab # [ 2.094741] netfs: FS-Cache loaded506gitlab # [ 2.101382] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...507gitlab # [ 2.103003] 9pnet: Installing 9P2000 support508gitlab # [ 2.107801] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.509gitlab # [ 2.109030] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.510gitlab # [ 2.111400] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev511gitlab # [ 1.968067] systemd-modules-load[94]: Using 4 probe threads512gitlab # [ 1.969393] systemd-modules-load[94]: Inserted module 'virtio_gpu'513gitlab # [ 2.115597] systemd[1]: Finished Load Kernel Module 9pnet_virtio.514gitlab # [ 2.117601] systemd[1]: Started Journal Service.515gitlab # [ 1.972484] systemd-modules-load[94]: Inserted module 'virtio_balloon'516gitlab # [ 1.973697] systemd-modules-load[94]: Inserted module 'dm_mod'517gitlab # [ 1.974700] systemd[1]: Finished Load Kernel Modules.518gitlab # [ 1.976192] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.519gitlab # [ 1.980753] systemd[1]: Starting Apply Kernel Variables...520gitlab # [ 1.982584] systemd[1]: Starting Create Static Device Nodes in /dev...521gitlab # [ 1.999288] systemd[1]: Finished Create Static Device Nodes in /dev.522gitlab # [ 2.000371] systemd[1]: Finished Apply Kernel Variables.523gitlab # [ 2.001243] systemd[1]: Reached target Preparation for Local File Systems.524gitlab # [ 2.002223] systemd[1]: Reached target Local File Systems.525gitlab # [ 2.003033] systemd[1]: Starting Create System Files and Directories...526gitlab # [ 2.003971] systemd[1]: Starting Rule-based Manager for Device Events and Files...527gitlab # [ 2.011971] systemd[1]: Finished Create System Files and Directories.528gitlab # [ 2.023786] systemd-udevd[113]: Using default interface naming scheme 'v261'.529gitlab # [ 2.034245] systemd[1]: Started Rule-based Manager for Device Events and Files.530gitlab # [ 2.057972] systemd[1]: Finished Coldplug All udev Devices.531gitlab # [ 2.059349] systemd[1]: Reached target System Initialization.532gitlab # [ 2.060636] systemd[1]: Reached target Basic System.533gitlab # [ 2.301246] uhci_hcd 0000:00:01.2: UHCI Host Controller534gitlab # [ 2.302311] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1535gitlab # [ 2.302656] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12536gitlab # [ 2.304907] uhci_hcd 0000:00:01.2: detected 2 ports537gitlab # [ 2.306078] serio: i8042 KBD port at 0x60,0x64 irq 1538gitlab # [ 2.307565] serio: i8042 AUX port at 0x60,0x64 irq 12539gitlab # [ 2.308671] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140540gitlab # [ 2.309760] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18541gitlab # [ 2.311035] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1542gitlab # [ 2.312375] usb usb1: Product: UHCI Host Controller543gitlab # [ 2.313178] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd544gitlab # [ 2.313996] usb usb1: SerialNumber: 0000:00:01.2545gitlab # [ 2.315603] SCSI subsystem initialized546gitlab # [ 2.321567] hub 1-0:1.0: USB hub found547gitlab # [ 2.322494] virtio_blk virtio4: 4/0/0 default/read/poll queues548gitlab # [ 2.322700] hub 1-0:1.0: 2 ports detected549gitlab # [ 2.327856] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)550gitlab # [ 2.374582] scsi host0: ata_piix551gitlab # [ 2.375625] scsi host1: ata_piix552gitlab # [ 2.382495] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0553gitlab # [ 2.384029] virtio_blk virtio5: 4/0/0 default/read/poll queues554gitlab # [ 2.238397] (udev-worker)[121]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.555gitlab # [ 2.386467] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0556gitlab # [ 2.242463] (udev-worker)[121]: Network interface NamePolicy= disabled on kernel command line.557gitlab # [ 2.243916] systemd[1]: Starting Virtual Cons[ 2.389976] virtio_blk virtio5: [vdb] 11843144 512-byte logical blocks (6.06 GB/5.65 GiB)558gitlab # ole Setup...559gitlab # [ 2.247080] (udev-worker)[140]: Network interface NamePolicy= disabled on kernel command line.560gitlab # [ 2.263256] systemd-vconsole-setup[151]: Configuration of first virtual console was skipped, ignoring remaining ones.561gitlab # [ 2.264922] systemd[1]: Finished Virtual Console Setup.562gitlab # [ 2.547782] ata2: found unknown device (class 0)563gitlab # [ 2.550494] usb 1-1: new full-speed USB device number 2 using uhci_hcd564gitlab # [ 2.550914] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100565gitlab # [ 2.557417] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5566gitlab # [ 2.587127] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0567gitlab # [ 2.465832] systemd[1]: Found device /dev/disk/by-label/nix-store.568gitlab # [ 2.617729] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray569gitlab # [ 2.472339] systemd[1]: Found device /dev/disk/by-label/nixos.570gitlab # [ 2.476381] systemd[1]: Reached target Initrd Root Device.571gitlab # [ 2.479168] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...572gitlab # [ 2.659254] cdrom: Uniform CD-ROM driver Revision: 3.20573gitlab # [ 2.519265] systemd-fsck[159]: nixos: clean, 12/65536 files, 13019/262144 blocks574gitlab # [ 2.527183] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.575gitlab # [ 2.716199] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00576gitlab # [ 2.719577] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10577gitlab # [ 2.722225] usb 1-1: Product: QEMU USB Tablet578gitlab # [ 2.723900] usb 1-1: Manufacturer: QEMU579gitlab # [ 2.725416] usb 1-1: SerialNumber: 28754-0000:00:01.2-1580gitlab # [ 2.756251] hid: raw HID events driver (C) Jiri Kosina581gitlab # [ 2.769931] usbcore: registered new interface driver usbhid582gitlab # [ 2.771969] usbhid: USB HID core driver583gitlab # [ 2.780847] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2584gitlab # [ 2.783975] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0585gitlab # [ 2.917513] systemd[1]: Mounting /sysroot...586gitlab # [ 3.217260] EXT4-fs (vda): mounted filesystem ef5f3dc4-b9b2-4c64-b653-4134e059cbfb r/w with ordered data mode. Quota mode: none.587gitlab # [ 3.077978] systemd[1]: Mounted /sysroot.588gitlab # [ 3.080580] systemd[1]: Reached target Initrd Root File System.589gitlab # [ 3.085667] systemd[1]: Mounting /sysroot/nix/.ro-store...590gitlab # [ 3.089377] systemd[1]: Mounting /sysroot/nix/.rw-store...591gitlab # [ 3.099476] systemd[1]: Mounting /sysroot/run...592gitlab # [ 3.101963] systemd[1]: Mounting /sysroot/tmp/shared...593gitlab # [ 3.105943] systemd[1]: Mounting /sysroot/tmp/xchg...594gitlab # [ 3.122303] systemd[1]: Starting Mountpoints Configured in the Real Root...595gitlab # [ 3.270515] erofs (device vdb): mounted with root inode @ nid 128.596gitlab # [ 3.126633] systemd[1]: Mounted /sysroot/nix/.ro-store.597gitlab # [ 3.127882] systemd[1]: Mounted /sysroot/nix/.rw-store.598gitlab # [ 3.129060] systemd[1]: Mounted /sysroot/run.599gitlab # [ 3.276909] 9p: Installing v9fs 9p2000 file system support600gitlab # [ 3.135610] systemd[1]: Starting rw-sysroot-nix-store.service...601gitlab # [ 3.138257] systemd[1]: Mounted /sysroot/tmp/xchg.602gitlab # [ 3.140166] systemd-sysroot-fstab-check[181]: /sysroot should be mounted in the initrd, will request daemon-reload.603gitlab # [ 3.142962] systemd[1]: Mounted /sysroot/tmp/shared.604gitlab # [ 3.144249] systemd[1]: Reload requested from client PID 181 ('systemd-sysroot') (unit initrd-parse-etc.service)...605gitlab # [ 3.146844] systemd[1]: Reloading...606gitlab # [ 3.197905] systemd[1]: Reloading finished in 55 ms.607gitlab # [ 3.218446] systemd-sysroot-fstab-check[181]: Requesting initrd-fs.target/start/replace...608gitlab # [ 3.221314] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.609gitlab # [ 3.223763] systemd[1]: Finished rw-sysroot-nix-store.service.610gitlab # [ 3.225887] systemd-sysroot-fstab-check[181]: Requesting swap.target/start/replace...611gitlab # [ 3.228440] systemd[1]: Starting rw-sysroot-nix-store.service...612gitlab # [ 3.230412] systemd[1]: initrd-parse-etc.service: Deactivated successfully.613gitlab # [ 3.232639] systemd[1]: Finished Mountpoints Configured in the Real Root.614gitlab # [ 3.234847] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.615gitlab # [ 3.261786] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.616gitlab # [ 3.264596] systemd[1]: Finished rw-sysroot-nix-store.service.617gitlab # [ 3.917896] systemd[1]: Mounting /sysroot/nix/store...618gitlab # [ 3.975787] systemd[1]: Mounted /sysroot/nix/store.619gitlab # [ 3.977647] systemd[1]: Reached target Initrd File Systems.620gitlab # [ 3.979571] systemd[1]: Starting Find NixOS closure...621gitlab # [ 3.981372] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...622gitlab # [ 4.006263] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.623gitlab # [ 4.008987] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.624gitlab # [ 4.028603] systemd[1]: Finished Find NixOS closure.625gitlab # [ 4.030430] systemd[1]: Reached target Initrd Default Target.626gitlab # [ 4.032388] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...627gitlab # [ 4.061995] systemd[1]: Stopped target Initrd Default Target.628gitlab # [ 4.064139] systemd[1]: Stopped target Basic System.629gitlab # [ 4.065877] systemd[1]: Stopped target Initrd Root Device.630gitlab # [ 4.067682] systemd[1]: Stopped target Path Units.631gitlab # [ 4.069382] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.632gitlab # [ 4.071845] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.633gitlab # [ 4.074412] systemd[1]: Stopped target Slice Units.634gitlab # [ 4.076148] systemd[1]: Stopped target Socket Units.635gitlab # [ 4.077875] systemd[1]: Stopped target System Initialization.636gitlab # [ 4.079803] systemd[1]: Stopped target Swaps.637gitlab # [ 4.081394] systemd[1]: Stopped target Timer Units.638gitlab # [ 4.083097] systemd[1]: dbus.socket: Deactivated successfully.639gitlab # [ 4.085686] systemd[1]: Closed D-Bus System Message Bus Socket.640gitlab # [ 4.088826] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.641gitlab # [ 4.091780] systemd[1]: Stopped Find NixOS closure.642gitlab # [ 4.093797] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio643gitlab # [ 4.097881] systemd[1]: Starting rw-sysroot-nix-store.service...644gitlab # [ 4.100382] systemd[1]: systemd-sysctl.service: Deactivated successfully.645gitlab # [ 4.102582] systemd[1]: Stopped Apply Kernel Variables.646gitlab # [ 4.104128] systemd[1]: systemd-modules-load.service: Deactivated successfully.647gitlab # [ 4.106397] systemd[1]: Stopped Load Kernel Modules.648gitlab # [ 4.107937] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.649gitlab # [ 4.110131] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.650gitlab # [ 4.112237] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.651gitlab # [ 4.114186] systemd[1]: Stopped Create System Files and Directories.652gitlab # [ 4.115862] systemd[1]: Stopped target Local File Systems.653gitlab # [ 4.117404] systemd[1]: Stopped target Preparation for Local File Systems.654gitlab # [ 4.119232] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.655gitlab # [ 4.121202] systemd[1]: Stopped Coldplug All udev Devices.656gitlab # [ 4.122533] systemd[1]: Stopping Rule-based Manager for Device Events and Files...657gitlab # [ 4.123830] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.658gitlab # [ 4.125174] systemd[1]: Stopped Virtual Console Setup.659gitlab # [ 4.126173] systemd[1]: initrd-cleanup.service: Deactivated successfully.660gitlab # [ 4.127324] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.661gitlab # [ 4.128227] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.662gitlab # [ 4.129198] systemd[1]: Finished rw-sysroot-nix-store.service.663gitlab # [ 4.130035] systemd[1]: systemd-udevd.service: Deactivated successfully.664gitlab # [ 4.130967] systemd[1]: Stopped Rule-based Manager for Device Events and Files.665gitlab # [ 4.131917] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.666gitlab # [ 4.132936] systemd[1]: Closed udev Control Socket.667gitlab # [ 4.133632] systemd[1]: Starting Cleanup udev Database...668gitlab # [ 4.134397] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.669gitlab # [ 4.135476] systemd[1]: Stopped Create Static Device Nodes in /dev.670gitlab # [ 4.136502] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.671gitlab # [ 4.137625] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.672gitlab # [ 4.138627] systemd[1]: kmod-static-nodes.service: Deactivated successfully.673gitlab # [ 4.139620] systemd[1]: Stopped Create List of Static Device Nodes.674gitlab # [ 4.153479] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.675gitlab # [ 4.154561] systemd[1]: Finished Cleanup udev Database.676gitlab # [ 4.155741] systemd[1]: Reached target Switch Root.677gitlab # [ 4.157805] systemd[1]: Starting NixOS Activation...678gitlab # [ 4.287345] initrd-nixos-activation-start[228]: booting system configuration /nix/store/wlx59kvgfj7m7h8yjhv348h405rq3xx9-nixos-system-gitlab-test679gitlab # [ 4.334078] initrd-nixos-activation-start[228]: running activation script...680gitlab # [ 4.572969] initrd-nixos-activation-start[251]: setting up /etc...681gitlab # [ 4.686170] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.682gitlab # [ 4.687823] systemd[1]: Finished NixOS Activation.683gitlab # [ 4.689620] systemd[1]: Starting Switch Root...684gitlab # [ 4.715703] systemd[1]: Switching root.685gitlab # [ 4.919412] systemd-journald[90]: Received SIGTERM from PID 1 (systemd).686gitlab # [ 5.039410] NET: Registered PF_VSOCK protocol family687gitlab # [ 5.411522] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)688gitlab # [ 5.421753] systemd[1]: Detected virtualization kvm.689gitlab # [ 5.423613] systemd[1]: Detected architecture x86-64.690gitlab # [ 5.425601] systemd[1]: Detected first boot.691gitlab # [ 5.428165] systemd[1]: Initializing machine ID from random generator.692gitlab # [ 5.571150] systemd[1]: bpf-restrict-fs: LSM BPF program attached693gitlab # [ 5.643661] systemd[1]: Applying preset policy.694gitlab # [ 5.796965] systemd[1]: Populated /etc with preset unit settings.695gitlab # [ 5.903466] systemd[1]: initrd-switch-root.service: Deactivated successfully.696gitlab # [ 5.904807] systemd[1]: Stopped initrd-switch-root.service.697gitlab # [ 5.907052] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.698gitlab # [ 5.909384] systemd[1]: Created slice Slice /system/getty.699gitlab # [ 5.910776] systemd[1]: Created slice GitLab DevOps Platform Slice.700gitlab # [ 5.912268] systemd[1]: Created slice User and Session Slice.701gitlab # [ 5.913196] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.702gitlab # [ 5.914473] systemd[1]: Started Forward Password Requests to Wall Directory Watch.703gitlab # [ 5.915723] systemd[1]: Expecting device /dev/hvc0...704gitlab # [ 5.916486] systemd[1]: Expecting device /dev/ttyS0...705gitlab # [ 5.917268] systemd[1]: Reached target Local Encrypted Volumes.706gitlab # [ 5.918107] systemd[1]: Stopped target initrd-fs.target.707gitlab # [ 5.918876] systemd[1]: Stopped target initrd-root-fs.target.708gitlab # [ 5.919776] systemd[1]: Stopped target initrd-switch-root.target.709gitlab # [ 5.920720] systemd[1]: Reached target Virtual Machines and Containers.710gitlab # [ 5.921670] systemd[1]: Reached target Path Units.711gitlab # [ 5.922415] systemd[1]: Reached target Remote File Systems.712gitlab # [ 5.923184] systemd[1]: Reached target Slice Units.713gitlab # [ 5.923941] systemd[1]: Reached target Swaps.714gitlab # [ 5.925452] systemd[1]: Listening on Query the User Interactively for a Password.715gitlab # [ 5.927490] systemd[1]: Listening on Process Core Dump Socket.716gitlab # [ 5.929018] systemd[1]: Listening on Credential Encryption/Decryption.717gitlab # [ 5.930805] systemd[1]: Listening on Factory Reset Management.718gitlab # [ 5.931808] systemd[1]: Listening on Hostname Service Socket.719gitlab # [ 5.934223] systemd[1]: Starting Journal Log Access Socket...720gitlab # [ 5.935739] systemd[1]: Listening on Journal Audit Socket.721gitlab # [ 5.937819] systemd[1]: Listening on Console Output Muting Service Socket.722gitlab # [ 5.939054] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.723gitlab # [ 5.940210] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os724gitlab # [ 5.941802] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki725gitlab # [ 5.944508] systemd[1]: Listening on Disk Repartitioning Service Socket.726gitlab # [ 5.945622] systemd[1]: Listening on udev Control Socket.727gitlab # [ 5.946499] systemd[1]: Listening on udev Varlink Socket.728gitlab # [ 5.949555] systemd[1]: Mounting Huge Pages File System...729gitlab # [ 5.951589] systemd[1]: Mounting POSIX Message Queue File System...730gitlab # [ 5.953626] systemd[1]: Mounting Kernel Debug File System...731gitlab # [ 5.957396] systemd[1]: Mounting Kernel Trace File System...732gitlab # [ 5.962821] systemd[1]: Starting Create List of Static Device Nodes...733gitlab # [ 5.966183] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio734gitlab # [ 5.979703] systemd[1]: Starting Load Kernel Module configfs...735gitlab # [ 5.980686] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm736gitlab # [ 5.982483] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore737gitlab # [ 5.984139] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse738gitlab # [ 5.991501] systemd[1]: Mounting FUSE Control File System...739gitlab # [ 5.992447] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67740gitlab # [ 5.996671] systemd[1]: Starting Journal Service...741gitlab # [ 6.014686] systemd[1]: Starting Load Kernel Modules...742gitlab # [ 6.019119] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...743gitlab # [ 6.023896] systemd[1]: Starting Remount Root and Kernel File Systems...744gitlab # [ 6.024935] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os745gitlab # [ 6.028084] systemd[1]: Starting Coldplug All udev Devices...746gitlab # [ 6.034485] systemd[1]: Listening on Journal Log Access Socket.747gitlab # [ 6.036411] systemd[1]: Mounted Huge Pages File System.748gitlab # [ 6.037426] systemd[1]: Mounted POSIX Message Queue File System.749gitlab # [ 6.038508] systemd[1]: Mounted Kernel Debug File System.750gitlab # [ 6.039519] systemd[1]: Mounted Kernel Trace File System.751gitlab # [ 6.043089] systemd[1]: Finished Create List of Static Device Nodes.752gitlab # [ 6.043718] systemd-journald[321]: Collecting audit messages is enabled.753gitlab # [ 6.045851] systemd[1]: modprobe@configfs.service: Deactivated successfully.754gitlab # [ 6.046776] loop: module loaded755gitlab # [ 6.050569] EXT4-fs (vda): re-mounted ef5f3dc4-b9b2-4c64-b653-4134e059cbfb.756gitlab # [ 6.053994] systemd[1]: Finished Load Kernel Module configfs.757gitlab # [ 6.055905] systemd[1]: Mounted FUSE Control File System.758gitlab # [ 6.057229] systemd[1]: Finished Load Kernel Modules.759gitlab # [ 6.058610] systemd[1]: Finished Remount Root and Kernel File Systems.760gitlab # [ 6.060104] systemd[1]: Listening on Disk Image Download Service Socket.761gitlab # [ 6.064432] systemd[1]: Mounting Kernel Configuration File System...762gitlab # [ 5.921214] systemd[1]: Queued start job for default target Multi-User System.763gitlab # [ 5.922506] systemd[1]: Unnecessary job was removed for /dev/vdb.764gitlab # [ 6.069604] systemd[1]: Starting Firewall...765gitlab # [ 6.070368] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore766gitlab # [ 5.926174] systemd[1]: systemd-journald.service: Deactivated successfully.767gitlab # [ 5.927512] systemd-modules-load[323]: Using 3 probe threads768gitlab # [ 5.928511] systemd-modules-load[323]: Inserted module 'tls'769gitlab # [ 5.929442] systemd-modules-load[323]: Inserted module 'loop'770gitlab # [ 5.930602] systemd-oomd[324]: No swap; memory pressure usage will be degraded771gitlab # [ 6.080243] systemd[1]: Starting Load/Save OS Random Seed...772gitlab # [ 6.082082] systemd[1]: Starting Apply Kernel Variables...773gitlab # [ 6.084831] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...774gitlab # [ 6.086011] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os775gitlab # [ 6.088460] systemd[1]: Started Journal Service.776gitlab # [ 5.943520] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.777gitlab # [ 5.944742] systemd[1]: Mounted Kernel Configuration File System.778gitlab # [ 5.954268] systemd[1]: Starting Flush Journal to Persistent Storage...779gitlab # [ 5.959962] systemd[1]: Finished Load/Save OS Random Seed.780gitlab # [ 5.962038] systemd[1]: Reached target First Boot Complete.781gitlab # [ 6.111433] systemd-journald[321]: Received client request to flush runtime journal.782gitlab # [ 6.004768] systemd[1]: Finished Apply Kernel Variables.783gitlab # [ 6.005909] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.784gitlab # [ 6.007397] systemd[1]: Starting Create Static Device Nodes in /dev...785gitlab # [ 6.008367] systemd[1]: Finished Create Static Device Nodes in /dev.786gitlab # [ 6.009298] systemd[1]: Reached target Preparation for Local File Systems.787gitlab # [ 6.010982] systemd[1]: Starting Rule-based Manager for Device Events and Files...788gitlab # [ 6.012064] systemd[1]: Finished Flush Journal to Persistent Storage.789gitlab # [ 6.018821] systemd-udevd[360]: Using default interface naming scheme 'v261'.790gitlab # [ 6.033451] systemd[1]: Finished Coldplug All udev Devices.791gitlab # [ 6.047544] systemd[1]: Started Rule-based Manager for Device Events and Files.792gitlab # [ 6.113527] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse793gitlab # [ 6.153749] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.794gitlab # [ 6.160631] (udev-worker)[435]: Network interface NamePolicy= disabled on kernel command line.795gitlab # [ 6.161865] systemd[1]: Finished Firewall.796gitlab # [ 6.165504] (udev-worker)[425]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.797gitlab # [ 6.168140] (udev-worker)[425]: Network interface NamePolicy= disabled on kernel command line.798gitlab # [ 6.170208] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.799gitlab # [ 6.190692] systemd[1]: Condition check resulted in Virtio network device being skipped.800gitlab # [ 6.191904] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore801gitlab # [ 6.193406] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67802gitlab # [ 6.195570] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore803gitlab # [ 6.197170] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os804gitlab # [ 6.198473] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os805gitlab # [ 6.363809] mousedev: PS/2 mouse device common for all mice806gitlab # [ 6.384469] parport_pc 00:03: reported by Plug and Play ACPI807gitlab # [ 6.385383] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)]808gitlab # [ 6.386413] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3809gitlab # [ 6.394348] ACPI: button: Power Button [PWRF]810gitlab # [ 6.396401] Floppy drive(s): fd0 is 2.88M AMI BIOS811gitlab # [ 6.397299] rtc_cmos 00:05: RTC can wake from S4812gitlab # [ 6.408048] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console813gitlab # [ 6.409691] FDC 0 is a S82078B814gitlab # [ 6.413747] rtc_cmos 00:05: registered as rtc0815gitlab # [ 6.413825] rtc_cmos 00:05: setting system clock to 2026-09-06T16:26:17 UTC (1788711977)816gitlab # [ 6.417993] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs817gitlab # [ 6.417994] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5818gitlab # [ 6.418092] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0819gitlab # [ 6.418634] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD820gitlab # [ 6.422372] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6821gitlab # [ 6.430056] Console: switching to colour dummy device 80x25822gitlab # [ 6.437062] [drm] Found bochs VGA, ID 0xb0c5.823gitlab # [ 6.437065] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.824gitlab # [ 6.437503] systemd-journald[321]: Time jumped backwards, rotating.825gitlab # [ 6.439660] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4826gitlab # [ 6.441886] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic827gitlab # [ 6.442867] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0828gitlab # [ 6.467059] Console: switching to colour frame buffer device 160x50829gitlab # [ 6.470915] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device830gitlab # [ 6.386338] systemd[1]: Starting Virtual Console Setup...831gitlab # [ 6.390693] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.832gitlab # [ 6.391796] systemd[1]: Stopped Virtual Console Setup.833gitlab # [ 6.395731] systemd[1]: Starting Virtual Console Setup...834gitlab # [ 6.403236] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.835gitlab # [ 6.404976] systemd[1]: Stopped Virtual Console Setup.836gitlab # [ 6.556929] ppdev: user-space parallel port driver837gitlab # [ 6.410866] systemd[1]: Starting Virtual Console Setup...838gitlab # [ 6.580290] kvm_amd: TSC scaling supported839gitlab # [ 6.580977] kvm_amd: Nested Virtualization enabled840gitlab # [ 6.581706] kvm_amd: Nested Paging enabled841gitlab # [ 6.582219] kvm_amd: LBR virtualization supported842gitlab # [ 6.582925] kvm_amd: Virtual VMLOAD VMSAVE supported843gitlab # [ 6.583621] kvm_amd: Virtual GIF supported844gitlab # [ 6.584098] kvm_amd: Virtual NMI enabled845gitlab # [ 6.615909] EDAC MC: Ver: 3.0.0846gitlab # [ 6.477040] systemd-vconsole-setup[495]: Configuration of first virtual console was skipped, ignoring remaining ones.847gitlab # [ 6.478733] systemd[1]: Finished Virtual Console Setup.848gitlab # [ 6.762279] systemd[1]: Mounting /run/wrappers...849gitlab # [ 6.787663] systemd[1]: Mounted /run/wrappers.850gitlab # [ 6.789402] systemd[1]: Reached target Local File Systems.851gitlab # [ 6.791295] systemd[1]: Listening on Boot Loader Control Service Socket.852gitlab # [ 6.793959] systemd[1]: Starting register-nix-paths.service...853gitlab # [ 6.796351] systemd[1]: Starting Create SUID/SGID Wrappers...854gitlab # [ 6.798473] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.855gitlab # [ 6.801446] systemd[1]: Starting Save Transient machine-id to Disk...856gitlab # [ 6.803582] systemd[1]: Starting Create System Files and Directories...857gitlab # [ 6.846205] systemd[1]: Finished Create System Files and Directories.858gitlab # [ 6.848726] systemd[1]: Starting Rebuild Journal Catalog...859gitlab # [ 6.851153] systemd[1]: Starting Record System Boot/Shutdown in UTMP...860gitlab # [ 6.870209] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.861gitlab # [ 6.874123] systemd[1]: Finished Save Transient machine-id to Disk.862gitlab # [ 6.878383] systemd[1]: Finished Record System Boot/Shutdown in UTMP.863gitlab # [ 6.893783] systemd[1]: Finished Rebuild Journal Catalog.864gitlab # [ 6.896613] systemd[1]: Starting Update is Completed...865gitlab # [ 6.914238] systemd[1]: Finished Update is Completed.866gitlab # [ 6.960716] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.867gitlab # [ 6.963452] systemd[1]: Finished Create SUID/SGID Wrappers.868gitlab # [ 7.206980] systemd[1]: Finished register-nix-paths.service.869gitlab # [ 7.209356] systemd[1]: Reached target System Initialization.870gitlab # [ 7.211380] systemd[1]: Started Discard unused filesystem blocks once a week.871gitlab # [ 7.213791] systemd[1]: Started logrotate.timer.872gitlab # [ 7.215473] systemd[1]: Started Daily Cleanup of Temporary Directories.873gitlab # [ 7.217699] systemd[1]: Reached target Timer Units.874gitlab # [ 7.219466] systemd[1]: Listening on D-Bus System Message Bus Socket.875gitlab # [ 7.221669] systemd[1]: Listening on Nix Daemon Socket.876gitlab # [ 7.223554] systemd[1]: Starting nixbot CI service socket...877gitlab # [ 7.225518] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.878gitlab # [ 7.228431] systemd[1]: Starting D-Bus System Message Bus...879gitlab # [ 7.230909] systemd[1]: Listening on nixbot CI service socket.880gitlab # [ 7.233076] systemd[1]: Reached target Socket Units.881gitlab # [ 7.272554] dbus-broker-launch[577]: Looking up NSS user entry for 'systemd-timesync'...882gitlab # [ 7.275741] dbus-broker-launch[577]: NSS returned no entry for 'systemd-timesync'883gitlab # [ 7.278587] dbus-broker-launch[577]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"884gitlab # [ 7.284795] systemd[1]: Started D-Bus System Message Bus.885gitlab # [ 7.298542] dbus-broker-launch[577]: Ready886gitlab # [ 7.302885] systemd[1]: Reached target Basic System.887gitlab # [ 7.305878] systemd[1]: Started backdoor.service.888gitlab # [ 7.308780] systemd[1]: Starting gitlab-config.service...889gitlab # [ 7.311577] systemd[1]: Starting Import lastlog data into lastlog2 database...890gitlab # [ 7.329764] systemd[1]: Starting Logrotate configuration check...891gitlab # [ 7.338507] systemd[1]: Starting Name Service Cache Daemon (nsncd)...892gitlab # [ 7.343569] systemd[1]: Starting Post-Boot Actions...893gitlab # [ 7.349935] systemd[1]: Started Reset console on configuration changes.894gitlab # [ 7.358309] logrotate[582]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.895gitlab # connecting to host...896gitlab # [ 7.362035] logrotate[582]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf897gitlab # [ 7.363310] logrotate[582]: note: 'monthly' overrides previously specified 'weekly'898gitlab # [ 7.363935] logrotate[582]: Reading state from file: /var/lib/logrotate.status899gitlab # [ 7.365700] logrotate[582]: state file /var/lib/logrotate.status does not exist900gitlab # [ 7.366905] logrotate[582]: Allocating hash table for state file, size 64 entries901gitlab # [ 7.367850] logrotate[582]: Handling 4 logs902gitlab: Guest shell says: b'Spawning backdoor root shell...\n'903gitlab # [ 7.369809] logrotate[582]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed904gitlab # [ 7.370868] logrotate[582]: considering log /var/log/btmp905gitlab: connected to guest root shell906gitlab # [ 7.372241] logrotate[582]: Creating new state907gitlab: (connecting took 8.00 seconds)908gitlab # [ 7.373172] logrotate[582]: Now: 2026-09-06 16:26909gitlab: (finished: waiting for the VM to finish booting, in 8.00 seconds)910gitlab # [ 7.374123] logrotate[582]: Last rotated at 2026-09-06 16:00911gitlab # [ 7.374422] logrotate[582]: log does not need rotating (log has already been rotated)912gitlab # [ 7.375697] logrotate[582]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed913gitlab # [ 7.376621] logrotate[582]: considering log /var/log/wtmp914gitlab # [ 7.377740] logrotate[582]: Creating new state915gitlab # [ 7.378607] logrotate[582]: Now: 2026-09-06 16:26916gitlab # [ 7.378937] logrotate[582]: Last rotated at 2026-09-06 16:00917gitlab # [ 7.379662] logrotate[582]: log does not need rotating (log has already been rotated)918gitlab # [ 7.380153] logrotate[582]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed919gitlab # [ 7.380577] logrotate[582]: switching euid from 0 to 165 and egid from 0 to 165 (pid 582)920gitlab # [ 7.381051] logrotate[582]: considering log /var/gitlab/state/log/*.log921gitlab # [ 7.381467] logrotate[582]: log /var/gitlab/state/log/*.log does not exist -- skipping922gitlab # [ 7.381908] logrotate[582]: Creating new state923gitlab # [ 7.382490] logrotate[582]: switching euid from 165 to 0 and egid from 165 to 0 (pid 582)924gitlab # [ 7.382962] logrotate[582]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed925gitlab # [ 7.383531] logrotate[582]: switching euid from 0 to 60 and egid from 0 to 60 (pid 582)926gitlab # [ 7.383972] logrotate[582]: considering log /var/log/nginx/*.log927gitlab # [ 7.384543] logrotate[582]: log /var/log/nginx/*.log does not exist -- skipping928gitlab # [ 7.384996] logrotate[582]: Creating new state929gitlab # [ 7.385426] logrotate[582]: not running postrotate script, since no logs were rotated930gitlab # [ 7.385857] logrotate[582]: switching euid from 60 to 0 and egid from 60 to 0 (pid 582)931gitlab # [ 7.386369] systemd[1]: Starting resolvconf update...932gitlab # [ 7.387139] nsncd[583]: Sep 06 16:26:18.612 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"933gitlab # [ 7.403123] systemd[1]: Started Name Service Cache Daemon (nsncd).934gitlab # [ 7.409650] systemd[1]: Finished Logrotate configuration check.935gitlab # [ 7.413112] systemd[1]: Finished Post-Boot Actions.936gitlab # [ 7.415285] systemd[1]: Finished Import lastlog data into lastlog2 database.937gitlab # [ 7.421098] systemd[1]: Reached target Host and Network Name Lookups.938gitlab # [ 7.421867] systemd[1]: Reached target User and Group Name Lookups.939gitlab # [ 7.426683] systemd[1]: Starting User Login Management...940gitlab # [ 7.460494] systemd[1]: Stopped target Host and Network Name Lookups.941gitlab # [ 7.461774] systemd[1]: Stopping Host and Network Name Lookups...942gitlab # [ 7.462737] systemd[1]: Stopped target User and Group Name Lookups.943gitlab # [ 7.463714] systemd[1]: Stopping User and Group Name Lookups...944gitlab # [ 7.464525] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...945gitlab # [ 7.465459] systemd[1]: nscd.service: Deactivated successfully.946gitlab # [ 7.466256] systemd[1]: Stopped Name Service Cache Daemon (nsncd).947gitlab # [ 7.467194] systemd[1]: nscd.service: Consumed 27ms CPU time over 128ms wall clock time, 7.5M memory peak, 5.5M read from disk.948gitlab # [ 7.468807] systemd[1]: Starting Name Service Cache Daemon (nsncd)...949gitlab # [ 7.472619] systemd-logind[618]: New seat seat0.950gitlab # [ 7.478717] systemd-logind[618]: Watching system buttons on /dev/input/event2 (Power Button)951gitlab # [ 7.481269] systemd-logind[618]: Watching system buttons on /dev/input/event4 (QEMU Virtio Keyboard)952gitlab # [ 7.482569] systemd-logind[618]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)953gitlab # [ 7.483897] systemd[1]: Started User Login Management.954gitlab # [ 7.484693] systemd[1]: Starting linger-users.service...955gitlab # [ 7.486386] nsncd[672]: Sep 06 16:26:18.718 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"956gitlab # [ 7.488521] systemd[1]: Started Name Service Cache Daemon (nsncd).957gitlab # [ 7.489504] systemd[1]: Reached target Host and Network Name Lookups.958gitlab # [ 7.490355] systemd[1]: Reached target User and Group Name Lookups.959gitlab # [ 7.500405] systemd[1]: linger-users.service: Deactivated successfully.960gitlab # [ 7.501396] systemd[1]: Finished linger-users.service.961gitlab # [ 7.504260] systemd[1]: Finished resolvconf update.962gitlab # [ 7.505103] systemd[1]: Reached target Preparation for Network.963gitlab # [ 7.506845] systemd[1]: Starting DHCP Client...964gitlab # [ 7.508794] systemd[1]: Starting Address configuration of eth1...965gitlab # [ 7.510309] systemd[1]: Starting Extra networking commands....966gitlab # [ 7.536724] network-addresses-eth1-start[700]: adding address 192.168.1.1/24... done967gitlab # [ 7.539920] network-addresses-eth1-start[700]: adding address 2001:db8:1::1/64... done968gitlab # [ 7.545084] systemd[1]: Finished Address configuration of eth1.969gitlab # [ 7.572671] dhcpcd[707]: dhcpcd-10.3.2 starting970gitlab # [ 7.574573] dhcpcd[762]: dev: loaded udev971gitlab # [ 7.578249] systemd[1]: Finished Extra networking commands..972gitlab # [ 7.579483] systemd[1]: Reached target Network.973gitlab # [ 7.726732] 8021q: 802.1Q VLAN Support v1.8974gitlab # [ 7.727217] 8021q: adding VLAN 0 to HW filter on device eth1975gitlab # [ 7.582071] systemd[1]: Starting gitlab-workhorse.service...976gitlab # [ 7.586623] systemd[1]: Starting Nginx Web Server...977gitlab # [ 7.589303] systemd[1]: Starting PostgreSQL Server...978gitlab # [ 7.593843] systemd[1]: Starting Redis Server - redis-gitlab...979gitlab # [ 7.596322] systemd[1]: Starting Permit User Sessions...980gitlab # [ 7.607031] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.981gitlab # [ 7.611335] systemd[1]: Finished Permit User Sessions.982gitlab # [ 7.613294] systemd[1]: Started Getty on tty1.983gitlab # [ 7.614433] systemd[1]: Reached target Login Prompts.984gitlab # [ 7.813444] cfg80211: Loading compiled-in X.509 certificates for regulatory database985gitlab # [ 7.821676] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'986gitlab # [ 7.822657] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'987gitlab # [ 7.825365] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2988gitlab # [ 7.826507] cfg80211: failed to load regulatory.db989gitlab # [ 7.855815] 8021q: adding VLAN 0 to HW filter on device eth0990gitlab # [ 7.710570] dhcpcd[762]: eth0: waiting for carrier991gitlab # [ 7.712083] dhcpcd[762]: eth0: carrier acquired992gitlab # [ 7.716409] dhcpcd[762]: DUID 00:01:00:01:32:30:50:aa:52:54:00:12:34:56993gitlab # [ 7.717539] dhcpcd[762]: eth0: IAID 00:12:34:56994gitlab # [ 7.719194] dhcpcd[762]: eth0: adding address fe80::5054:ff:fe12:3456995gitlab # [ 7.721239] redis-server[789]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.996gitlab # [ 7.722317] redis-server[789]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo997gitlab # [ 7.723767] redis-server[789]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=789, just started998gitlab # [ 7.725419] redis-server[789]: Configuration loaded999gitlab # [ 7.727457] nginx-pre-start[792]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok1000gitlab # [ 7.728250] nginx-pre-start[792]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful1001gitlab # [ 7.728978] systemd[1]: Started Redis Server - redis-gitlab.1002gitlab # [ 7.729624] redis[789]: monotonic clock: POSIX clock_gettime1003gitlab # [ 7.730476] redis[789]: Running mode=standalone, port=0.1004gitlab # [ 7.731335] redis[789]: Server initialized1005gitlab # [ 7.731978] systemd[1]: Started Nginx Web Server.1006gitlab # [ 7.732857] redis[789]: Ready to accept connections unix1007gitlab # [ 7.732960] redis[789]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1008gitlab # [ 7.743356] postgresql-pre-start[797]: The files belonging to this database system will be owned by user "postgres".1009gitlab # [ 7.744260] postgresql-pre-start[797]: This user must also own the server process.1010gitlab # [ 7.748153] postgresql-pre-start[797]: The database cluster will be initialized with locale "en_US.UTF-8".1011gitlab # [ 7.749650] postgresql-pre-start[797]: The default database encoding has accordingly been set to "UTF8".1012gitlab # [ 7.750873] postgresql-pre-start[797]: The default text search configuration will be set to "english".1013gitlab # [ 7.752382] postgresql-pre-start[797]: Data page checksums are enabled.1014gitlab # [ 7.752904] postgresql-pre-start[797]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok1015gitlab # [ 7.755524] postgresql-pre-start[797]: creating subdirectories ... ok1016gitlab # [ 7.756400] postgresql-pre-start[797]: selecting dynamic shared memory implementation ... posix1017gitlab # [ 7.784306] postgresql-pre-start[797]: selecting default "max_connections" ... 1001018gitlab # [ 7.820073] postgresql-pre-start[797]: selecting default "shared_buffers" ... 128MB1019gitlab # [ 7.856972] dhcpcd[762]: eth0: soliciting a DHCP lease1020gitlab # [ 8.009336] NET: Registered PF_PACKET protocol family1021gitlab # [ 7.872620] dhcpcd[762]: eth0: offered 10.0.2.15 from 10.0.2.21022gitlab # [ 7.875274] dhcpcd[762]: eth0: probing address 10.0.2.15/241023gitlab # [ 8.104432] systemd[1]: Started gitlab-workhorse.service.1024gitlab # [ 8.224339] postgresql-pre-start[797]: selecting default time zone ... UTC1025gitlab # [ 8.225918] postgresql-pre-start[797]: creating configuration files ... ok1026gitlab # [ 8.292336] gitlab-workhorse[826]: time="2026-09-06T16:26:19Z" level=info msg=Starting build_time=19700101.000000 version=19.0.41027gitlab # [ 8.293191] gitlab-workhorse[826]: time="2026-09-06T16:26:19Z" level=info msg="Using redis/go-redis"1028gitlab # [ 8.293866] gitlab-workhorse[826]: time="2026-09-06T16:26:19Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1029gitlab # [ 8.393031] postgresql-pre-start[797]: running bootstrap script ... ok1030gitlab # [ 8.802888] postgresql-pre-start[797]: performing post-bootstrap initialization ... ok1031gitlab # [ 9.065743] postgresql-pre-start[797]: syncing data to disk ... ok1032gitlab # [ 9.067024] postgresql-pre-start[797]: initdb: warning: enabling "trust" authentication for local connections1033gitlab # [ 9.067695] postgresql-pre-start[797]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1034gitlab # [ 9.069355] postgresql-pre-start[797]: Success. You can now start the database server using:1035gitlab # [ 9.070831] postgresql-pre-start[797]: pg_ctl -D /var/lib/postgresql/18 -l logfile start1036gitlab # [ 9.177872] postgres[851]: [851] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1037gitlab # [ 9.179148] postgres[851]: [851] LOG: listening on IPv6 address "::1", port 54321038gitlab # [ 9.180052] postgres[851]: [851] LOG: listening on IPv4 address "127.0.0.1", port 54321039gitlab # [ 9.183075] postgres[851]: [851] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1040gitlab # [ 9.197900] postgres[860]: [860] LOG: database system was shut down at 2026-09-06 16:26:20 GMT1041gitlab # [ 9.206805] postgres[851]: [851] LOG: database system is ready to accept connections1042gitlab # [ 9.213601] systemd[1]: Started PostgreSQL Server.1043gitlab # [ 9.218108] systemd[1]: Starting PostgreSQL Setup Scripts...1044gitlab # [ 9.451563] postgresql-setup-start[871]: CREATE DATABASE1045gitlab # [ 9.476175] postgresql-setup-start[881]: CREATE ROLE1046gitlab # [ 9.490847] postgresql-setup-start[883]: ALTER DATABASE1047gitlab # [ 9.515293] postgresql-setup-start[892]: CREATE ROLE1048gitlab # [ 9.518338] systemd[1]: Finished PostgreSQL Setup Scripts.1049gitlab # [ 9.519729] systemd[1]: Reached target PostgreSQL.1050gitlab # [ 9.522210] systemd[1]: Starting gitlab-postgresql.service...1051gitlab # [ 9.571949] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[876]: mkdir -p /var/gitlab/state/home/.ssh: OK1052gitlab # [ 9.573402] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[876]: chmod 700 /var/gitlab/state/home/.ssh: OK1053gitlab # [ 9.604743] gitlab-postgresql-start[905]: CREATE DATABASE1054gitlab # [ 9.612080] systemd[1]: Finished gitlab-config.service.1055gitlab # [ 9.613933] systemd[1]: Started gitaly.service.1056gitlab # [ 9.616316] systemd[1]: Starting gitlab-db-config.service...1057gitlab # [ 9.632418] gitlab-postgresql-start[922]: CREATE EXTENSION1058gitlab # [ 9.709518] gitlab-postgresql-start[926]: CREATE EXTENSION1059gitlab # [ 9.712837] systemd[1]: Finished gitlab-postgresql.service.1060gitlab # [ 10.228905] gitaly[921]: time="2026-09-06T16:26:21.460Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=9211061gitlab # [ 10.230173] gitaly[921]: time="2026-09-06T16:26:21.461Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=9211062gitlab # [ 10.230839] gitaly[921]: time="2026-09-06T16:26:21.461Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=9211063gitlab # [ 10.232173] gitaly[921]: time="2026-09-06T16:26:21.461Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=9211064gitlab # [ 10.233093] gitaly[921]: time="2026-09-06T16:26:21.461Z" level=info msg="Starting Gitaly" pid=921 version=1065gitlab # [ 10.403675] dhcpcd[762]: eth0: soliciting an IPv6 router1066gitlab # [ 10.404631] dhcpcd[762]: eth0: Router Advertisement from fe80::21067gitlab # [ 10.405568] dhcpcd[762]: eth0: adding address fec0::5054:ff:fe12:3456/641068gitlab # [ 10.406647] dhcpcd[762]: eth0: adding route to fec0::/641069gitlab # [ 10.407644] dhcpcd[762]: eth0: adding default route via fe80::21070gitlab # [ 10.421426] gitaly[921]: time="2026-09-06T16:26:21.653Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=191 pid=9211071gitlab # [ 10.422030] gitaly[921]: time="2026-09-06T16:26:21.653Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=9211072gitlab # [ 10.422557] gitaly[921]: time="2026-09-06T16:26:21.653Z" level=info msg="finished initializing command factory" duration_ms=0 pid=9211073gitlab # [ 10.423228] gitaly[921]: time="2026-09-06T16:26:21.653Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-921/git-exec-3434367360.d/git pid=9211074gitlab # [ 10.434129] gitaly[921]: time="2026-09-06T16:26:21.666Z" level=info msg="finished detecting git version" duration_ms=12 pid=9211075gitlab # [ 10.435938] gitaly[921]: time="2026-09-06T16:26:21.666Z" level=info msg="using Git version" pid=921 version=2.53.g1076gitlab # [ 10.437822] gitaly[921]: time="2026-09-06T16:26:21.666Z" level=info msg="clearing disk cache object folder" pid=921 storage=default1077gitlab # [ 10.438632] gitaly[921]: time="2026-09-06T16:26:21.666Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1927961213 pid=921 storage=default1078gitlab # [ 10.439483] gitaly[921]: time="2026-09-06T16:26:21.666Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1927961213 pid=921 storage=default1079gitlab # [ 10.439988] gitaly[921]: time="2026-09-06T16:26:21.667Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=9211080gitlab # [ 10.440929] gitaly[921]: time="2026-09-06T16:26:21.667Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=9211081gitlab # [ 10.441923] gitaly[921]: time="2026-09-06T16:26:21.667Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1927961213 clear_duration_ms=0 pid=921 storage=default1082gitlab # [ 10.470194] gitaly[921]: time="2026-09-06T16:26:21.702Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=9211083gitlab # [ 10.471134] gitaly[921]: time="2026-09-06T16:26:21.702Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=921 schema=unix1084gitlab # [ 10.471879] gitaly[921]: time="2026-09-06T16:26:21.702Z" level=info msg="listening at address" address=/run/gitaly/gitaly-921/sock.d/intern pid=921 schema=unix1085gitlab # [ 10.472522] gitaly[921]: time="2026-09-06T16:26:21.702Z" level=info msg="counting repositories" pid=9211086gitlab # [ 10.473246] gitaly[921]: time="2026-09-06T16:26:21.703Z" level=info msg="starting to count repositories" pid=921 storage_path=/var/gitlab/state/repositories1087gitlab # [ 10.473740] gitaly[921]: time="2026-09-06T16:26:21.703Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=921 storage_path=/var/gitlab/state/repositories1088gitlab # [ 10.474641] gitaly[921]: time="2026-09-06T16:26:21.703Z" level=info msg="completed counting all repositories" counting_duration=0 pid=9211089gitlab # [ 10.484373] gitaly[921]: time="2026-09-06T16:26:21.716Z" level=info msg="finished tempdir cleaner walk" pid=921 storage=default time_ms=01090gitlab # [ 10.486839] gitaly[921]: time="2026-09-06T16:26:21.719Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9211091gitlab # [ 10.487644] gitaly[921]: time="2026-09-06T16:26:21.719Z" level=info msg="Started Gitaly" duration_ms=257 pid=9211092gitlab # [ 10.488689] gitaly[921]: time="2026-09-06T16:26:21.719Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9211093gitlab # [ 10.489646] gitaly[921]: time="2026-09-06T16:26:21.719Z" level=info msg="maintenance: daily scheduled" pid=921 scheduled="2026-09-07 12:00:00 +0000 UTC"1094gitlab # [ 11.101147] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[927]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1095gitlab # [ 11.102602] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[927]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1096gitlab # [ 11.130858] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[927]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1097gitlab # [ 12.379098] gitaly[921]: time="2026-09-06T16:26:23.610Z" level=info msg="License database preloaded" duration_ms=2148 pid=9211098gitlab # [ 12.828723] dhcpcd[762]: eth0: leased 10.0.2.15 for 86400 seconds1099gitlab # [ 12.830396] dhcpcd[762]: eth0: adding route to 10.0.2.0/241100gitlab # [ 12.831506] dhcpcd[762]: eth0: adding default route via 10.0.2.21101gitlab # [ 12.950667] systemd[1]: Started DHCP Client.1102gitlab # [ 12.954108] systemd[1]: Reached target Network is Online.1103gitlab # [ 42.174875] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[927]: W, [2026-09-06T16:26:53.406553 #927] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1104gitlab # [ 44.359825] postgres[1327]: [1327] ERROR: relation "feature_gates" does not exist at character 5231105gitlab # [ 44.360949] postgres[1327]: [1327] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1106gitlab # [ 44.361654] postgres[1327]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1107gitlab # [ 44.362828] postgres[1327]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1108gitlab # [ 44.363825] postgres[1327]: attidentity AS identity,1109gitlab # [ 44.364508] postgres[1327]: attgenerated as attgenerated1110gitlab # [ 44.365611] postgres[1327]: FROM pg_attribute a1111gitlab # [ 44.366508] postgres[1327]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1112gitlab # [ 44.367255] postgres[1327]: LEFT JOIN pg_type t ON a.atttypid = t.oid1113gitlab # [ 44.368092] postgres[1327]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1114gitlab # [ 44.368866] postgres[1327]: WHERE a.attrelid = '"feature_gates"'::regclass1115gitlab # [ 44.369903] postgres[1327]: AND a.attnum > 0 AND NOT a.attisdropped1116gitlab # [ 44.370836] postgres[1327]: ORDER BY a.attnum1117gitlab # [ 44.371590] postgres[1327]:1118gitlab # [ 48.090535] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1360]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1119gitlab # [ 48.091800] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1360]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1120gitlab # [ 48.358095] gitlab-workhorse[826]: time="2026-09-06T16:26:59Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1121gitlab # [ 58.369194] gitlab-workhorse[826]: time="2026-09-06T16:27:09Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1122gitlab # [ 63.724768] postgres[1435]: [1435] ERROR: relation "table_without_models" does not exist1123gitlab # [ 63.726375] postgres[1435]: [1435] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1124gitlab # [ 68.378773] gitlab-workhorse[826]: time="2026-09-06T16:27:19Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1125gitlab # [ 75.985909] systemd[1]: Finished gitlab-db-config.service.1126gitlab # [ 75.989807] systemd[1]: Started gitlab-sidekiq.service.1127gitlab # [ 75.992931] systemd[1]: Starting gitlab.service...1128gitlab # [ 76.357298] bundle[1611]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1129gitlab # [ 76.362128] bundle[1611]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1130gitlab # [ 76.366619] bundle[1611]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1131gitlab # [ 78.389469] gitlab-workhorse[826]: time="2026-09-06T16:27:29Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1132gitlab # [ 79.095285] sidekiq-cluster[1610]: {"severity":"INFO","time":"2026-09-06T16:27:30.310Z","message":"Starting cluster with 1 processes"}1133gitlab # [ 79.227802] bundle[1611]: {"timestamp":"2026-09-06T16:27:30.438Z","pid":1611,"message":"* Enabling systemd notification integration"}1134gitlab # [ 79.231694] bundle[1611]: {"timestamp":"2026-09-06T16:27:30.459Z","pid":1611,"message":"Puma starting in cluster mode..."}1135gitlab # [ 79.235194] bundle[1611]: {"timestamp":"2026-09-06T16:27:30.459Z","pid":1611,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1136gitlab # [ 79.238786] bundle[1611]: {"timestamp":"2026-09-06T16:27:30.459Z","pid":1611,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1137gitlab # [ 79.243180] bundle[1611]: {"timestamp":"2026-09-06T16:27:30.459Z","pid":1611,"message":"* Min threads: 0"}1138gitlab # [ 79.246307] bundle[1611]: {"timestamp":"2026-09-06T16:27:30.459Z","pid":1611,"message":"* Max threads: 4"}1139gitlab # [ 79.249266] bundle[1611]: {"timestamp":"2026-09-06T16:27:30.459Z","pid":1611,"message":"* Environment: production"}1140gitlab # [ 79.252527] bundle[1611]: {"timestamp":"2026-09-06T16:27:30.459Z","pid":1611,"message":"* Master PID: 1611"}1141gitlab # [ 79.255672] bundle[1611]: {"timestamp":"2026-09-06T16:27:30.459Z","pid":1611,"message":"* Workers: 2"}1142gitlab # [ 79.258705] bundle[1611]: {"timestamp":"2026-09-06T16:27:30.459Z","pid":1611,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1143gitlab # [ 79.262734] bundle[1611]: {"timestamp":"2026-09-06T16:27:30.459Z","pid":1611,"message":"* Preloading application"}1144gitlab # [ 88.390652] gitlab-workhorse[826]: time="2026-09-06T16:27:39Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1145gitlab # [ 98.400066] gitlab-workhorse[826]: time="2026-09-06T16:27:49Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1146gitlab # [ 102.990973] bundle[1611]: W, [2026-09-06T16:27:54.223006 #1611] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1147gitlab # [ 105.008190] sidekiq-cluster[1641]: 2026-09-06T16:27:56.239Z pid=1641 tid=wjl INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1641, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>436404224}1148gitlab # [ 105.189454] sidekiq-cluster[1641]: W, [2026-09-06T16:27:56.421649 #1641] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1149gitlab # [ 105.233849] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:56.465Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1150gitlab # [ 107.425082] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.656Z","message":"GitLab reliable fetch activated!"}1151gitlab # [ 107.426436] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.656Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1152gitlab # [ 107.449573] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.681Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1153gitlab # [ 107.451989] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.684Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1154gitlab # [ 107.454810] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.687Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1155gitlab # [ 107.457547] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.689Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1156gitlab # [ 107.460444] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.692Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1157gitlab # [ 107.463174] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.695Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1158gitlab # [ 107.465923] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.698Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1159gitlab # [ 107.468315] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.700Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1160gitlab # [ 107.470733] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.703Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1161gitlab # [ 107.472906] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.705Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1162gitlab # [ 107.475135] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.707Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1163gitlab # [ 107.477858] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.710Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1164gitlab # [ 107.479878] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.712Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1165gitlab # [ 107.482142] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.714Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1166gitlab # [ 107.484397] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.716Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1167gitlab # [ 107.486492] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.718Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1168gitlab # [ 107.489479] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.721Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1169gitlab # [ 107.492205] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.724Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1170gitlab # [ 107.495053] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.727Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1171gitlab # [ 107.497865] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.730Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1172gitlab # [ 107.500881] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.733Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1173gitlab # [ 107.503072] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.735Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1174gitlab # [ 107.505558] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.737Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1175gitlab # [ 107.508144] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.740Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1176gitlab # [ 107.510366] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.742Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1177gitlab # [ 107.513084] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.745Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1178gitlab # [ 107.516306] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.748Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1179gitlab # [ 107.524934] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.757Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1180gitlab # [ 107.532237] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.764Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1181gitlab # [ 107.538364] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.770Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1182gitlab # [ 107.544195] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.776Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1183gitlab # [ 107.549444] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.781Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1184gitlab # [ 107.554800] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.787Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1185gitlab # [ 107.559230] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.791Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1186gitlab # [ 107.563446] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.795Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1187gitlab # [ 107.567666] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.800Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1188gitlab # [ 107.572507] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.804Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1189gitlab # [ 107.575971] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.808Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1190gitlab # [ 107.579063] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.811Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1191gitlab # [ 107.582110] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.814Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1192gitlab # [ 107.585337] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.817Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1193gitlab # [ 107.588341] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.820Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1194gitlab # [ 107.591745] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.824Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1195gitlab # [ 107.597683] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.829Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1196gitlab # [ 107.602616] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.834Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1197gitlab # [ 107.606118] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.838Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1198gitlab # [ 107.609095] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.841Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1199gitlab # [ 107.612340] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.844Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1200gitlab # [ 107.615713] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.848Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1201gitlab # [ 107.618897] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.851Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1202gitlab # [ 107.622292] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.854Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1203gitlab # [ 107.625563] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.857Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1204gitlab # [ 107.635408] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.867Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1205gitlab # [ 107.639051] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.871Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1206gitlab # [ 107.642563] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.874Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1207gitlab # [ 107.646115] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.878Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1208gitlab # [ 107.649565] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.881Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1209gitlab # [ 107.653163] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.885Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1210gitlab # [ 107.655916] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.888Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1211gitlab # [ 107.658873] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.891Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1212gitlab # [ 107.661947] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.894Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1213gitlab # [ 107.664633] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.897Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1214gitlab # [ 107.668095] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.900Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1215gitlab # [ 107.671639] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.904Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1216gitlab # [ 107.675185] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.907Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1217gitlab # [ 107.678646] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.911Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1218gitlab # [ 107.681471] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.913Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1219gitlab # [ 107.684450] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.916Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1220gitlab # [ 107.687329] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.919Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1221gitlab # [ 107.689347] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.921Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1222gitlab # [ 107.691838] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.924Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1223gitlab # [ 107.693921] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.926Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1224gitlab # [ 107.696252] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.928Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1225gitlab # [ 107.698425] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.930Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1226gitlab # [ 107.700570] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.933Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1227gitlab # [ 107.702894] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.935Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1228gitlab # [ 107.705250] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.937Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1229gitlab # [ 107.707661] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.940Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1230gitlab # [ 107.709872] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.942Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1231gitlab # [ 107.712167] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.944Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1232gitlab # [ 107.714203] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.946Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1233gitlab # [ 107.716395] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.948Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1234gitlab # [ 107.719239] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.951Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1235gitlab # [ 107.721511] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.953Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1236gitlab # [ 107.723817] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.956Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1237gitlab # [ 107.726354] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.958Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1238gitlab # [ 107.728932] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.961Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1239gitlab # [ 107.731090] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.963Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1240gitlab # [ 107.733574] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.966Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1241gitlab # [ 107.735676] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.968Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1242gitlab # [ 107.737863] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.970Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1243gitlab # [ 107.740271] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.972Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1244gitlab # [ 107.742336] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.974Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1245gitlab # [ 107.744667] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.977Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1246gitlab # [ 107.747769] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.980Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1247gitlab # [ 107.750258] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.982Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1248gitlab # [ 107.752668] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.985Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1249gitlab # [ 107.754578] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.987Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1250gitlab # [ 107.764410] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.996Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1251gitlab # [ 107.766933] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:58.999Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1252gitlab # [ 107.769193] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:59.001Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1253gitlab # [ 107.811420] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:59.042Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1254gitlab # [ 107.814498] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:59.046Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1255gitlab # [ 107.817935] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:59.050Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1256gitlab # [ 107.821865] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:59.054Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1257gitlab # [ 107.825549] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:59.057Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1258gitlab # [ 107.828630] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:59.061Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1259gitlab # [ 107.832968] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:59.065Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1260gitlab # [ 107.837488] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:27:59.069Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1261gitlab # [ 108.410534] gitlab-workhorse[826]: time="2026-09-06T16:27:59Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1262gitlab # [ 118.412243] gitlab-workhorse[826]: time="2026-09-06T16:28:09Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1263gitlab # [ 128.421874] gitlab-workhorse[826]: time="2026-09-06T16:28:19Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1264gitlab # [ 130.316871] postgres[2098]: [2098] ERROR: relation "table_without_models" does not exist1265gitlab # [ 130.317655] postgres[2098]: [2098] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1266gitlab # [ 130.941381] postgres[2122]: [2122] ERROR: relation "table_without_models" does not exist1267gitlab # [ 130.944104] postgres[2122]: [2122] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1268gitlab # [ 132.774099] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:24.005Z","message":"Booted Rails 7.2.3.1 application in production environment"}1269gitlab # [ 132.778381] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:24.010Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1270gitlab # [ 132.783322] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:24.015Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1271gitlab # [ 132.787343] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:24.019Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1272gitlab # [ 132.789352] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:24.021Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1273gitlab # [ 133.869289] bundle[1611]: {"timestamp":"2026-09-06T16:28:25.100Z","pid":1611,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1274gitlab # [ 133.873601] bundle[1611]: {"timestamp":"2026-09-06T16:28:25.100Z","pid":1611,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1275gitlab # [ 133.877614] bundle[1611]: {"timestamp":"2026-09-06T16:28:25.100Z","pid":1611,"message":"! #\u003cThread:0x000075c7de95e108 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1276gitlab # [ 133.889154] bundle[1611]: {"timestamp":"2026-09-06T16:28:25.100Z","pid":1611,"message":"! #\u003cThread:0x000075c7debaa510 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1277gitlab # [ 133.900488] bundle[1611]: {"timestamp":"2026-09-06T16:28:25.101Z","pid":1611,"message":"Use Ctrl-C to stop"}1278gitlab # [ 133.903518] bundle[1611]: {"timestamp":"2026-09-06T16:28:25.101Z","pid":1611,"message":"Sending status to systemd every 1.0 sec"}1279gitlab # [ 134.239115] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:25.470Z","message":"Cleaning working queues"}1280gitlab # [ 134.245699] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:25.477Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"349259fbfe92959f2240d3fd","created_at":"2026-09-06T16:27:26.754Z","trace_propagation_headers":{"sentry-trace":"ff5996c6e34f43fca28de16cc3dd31ea-585f965f31324e7b","baggage":"sentry-trace_id=ff5996c6e34f43fca28de16cc3dd31ea,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"e724dd1acc121f98e4eb5e9ec025302f","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-06T16:27:26.761Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-349259fbfe92959f2240d3fd: start","job_status":"start","queue_duration_s":58.71586,"scheduling_latency_s":58.715935}1281gitlab # [ 134.276236] bundle[1611]: {"timestamp":"2026-09-06T16:28:25.488Z","pid":1611,"message":"- Worker 0 (PID: 2176) booted in 0.36s, phase: 0"}1282gitlab # [ 134.279502] bundle[1611]: {"timestamp":"2026-09-06T16:28:25.496Z","pid":1611,"message":"- Worker 1 (PID: 2180) booted in 0.34s, phase: 0"}1283gitlab # [ 134.282908] systemd[1]: Started gitlab.service.1284gitlab # [ 134.284338] systemd[1]: Reached target Common target for all GitLab services..1285gitlab # [ 134.286240] systemd[1]: Starting setup-gitlab.service...1286gitlab # [ 134.300826] setup-gitlab-start[2209]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1287gitlab # [ 134.415329] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:25.647Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"349259fbfe92959f2240d3fd","created_at":"2026-09-06T16:27:26.754Z","trace_propagation_headers":{"sentry-trace":"ff5996c6e34f43fca28de16cc3dd31ea-585f965f31324e7b","baggage":"sentry-trace_id=ff5996c6e34f43fca28de16cc3dd31ea,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"e724dd1acc121f98e4eb5e9ec025302f","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-06T16:27:26.761Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-349259fbfe92959f2240d3fd: done: 0.170115 sec","job_status":"done","queue_duration_s":58.71586,"scheduling_latency_s":58.715935,"redis_calls":24,"redis_duration_s":0.005601999999999999,"redis_read_bytes":912,"redis_write_bytes":2622,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.002678,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001839,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001085,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.058923,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.086748176,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.170115,"completed_at":"2026-09-06T16:28:25.647Z","load_balancing_strategy":"primary","db_duration_s":0.053844,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1288gitlab # [ 136.854112] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:28:28 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 25301289gitlab # [ 136.857862] setup-gitlab-start[2209]: + TOKEN=nixbot-test-token-of-20-chars1290gitlab # [ 136.858981] setup-gitlab-start[2209]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1291gitlab # [ 136.874569] runuser[2233]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1292gitlab # [ 137.258642] setup-gitlab-start[2234]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1293gitlab # [ 137.260279] setup-gitlab-start[2234]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1294gitlab # [ 137.261460] setup-gitlab-start[2234]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1295gitlab # [ 137.836819] setup-gitlab-start[2234]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1296gitlab # [ 137.838559] setup-gitlab-start[2234]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1297gitlab # [ 137.839937] setup-gitlab-start[2234]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1298gitlab # [ 148.972898] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:40.204Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"23c8c0d4845b3dd353de64ef","created_at":"2026-09-06T16:28:40.171Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7d780e740df9d41179d6d16e6f2ee6d5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.201Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-23c8c0d4845b3dd353de64ef: start","job_status":"start","queue_duration_s":0.002365,"scheduling_latency_s":0.002407}1299gitlab # [ 149.012573] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:40.244Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1300gitlab # [ 149.053639] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:40.285Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"23c8c0d4845b3dd353de64ef","created_at":"2026-09-06T16:28:40.171Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7d780e740df9d41179d6d16e6f2ee6d5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.201Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-23c8c0d4845b3dd353de64ef: done: 0.081238 sec","job_status":"done","queue_duration_s":0.002365,"scheduling_latency_s":0.002407,"redis_calls":10,"redis_duration_s":0.019843,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001423,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.017654,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000766,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001004597,"duration_s":0.081238,"completed_at":"2026-09-06T16:28:40.285Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010204,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1301gitlab # [ 149.100494] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:40.292Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"17ce72005883be03ff28bc55","created_at":"2026-09-06T16:28:40.212Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9a7f713a908dd77f461b5ffe60f76c0c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:28:40.228Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-17ce72005883be03ff28bc55: start","job_status":"start","queue_duration_s":0.06414,"scheduling_latency_s":0.064171}1302gitlab # [ 149.101968] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:40.324Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1303gitlab # [ 149.148037] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:40.379Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"17ce72005883be03ff28bc55","created_at":"2026-09-06T16:28:40.212Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9a7f713a908dd77f461b5ffe60f76c0c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:28:40.228Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-17ce72005883be03ff28bc55: done: 0.087567 sec","job_status":"done","queue_duration_s":0.06414,"scheduling_latency_s":0.064171,"redis_calls":10,"redis_duration_s":0.049455,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002856,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.041915,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004684,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015506,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002831924,"duration_s":0.087567,"completed_at":"2026-09-06T16:28:40.379Z","load_balancing_strategy":"primary","db_duration_s":0.010438,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1304gitlab # [ 149.169561] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:40.389Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"46543df3dbd82406fe9d1ca5","created_at":"2026-09-06T16:28:40.236Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb513cae34b7e0de521d20477234c9c1","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.255Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-46543df3dbd82406fe9d1ca5: start","job_status":"start","queue_duration_s":0.133422,"scheduling_latency_s":0.133464}1305gitlab # [ 149.295623] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:40.527Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"46543df3dbd82406fe9d1ca5","created_at":"2026-09-06T16:28:40.236Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb513cae34b7e0de521d20477234c9c1","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.255Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-46543df3dbd82406fe9d1ca5: done: 0.138328 sec","job_status":"done","queue_duration_s":0.133422,"scheduling_latency_s":0.133464,"redis_calls":10,"redis_duration_s":0.0102,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001842,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005181,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003177,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.039,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00201059,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.138328,"completed_at":"2026-09-06T16:28:40.527Z","load_balancing_strategy":"primary","db_duration_s":0.048339,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1306gitlab # [ 149.322377] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:40.532Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"54eae7ed963686556592ffbb","created_at":"2026-09-06T16:28:40.281Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef9f9801542ca796328eb484f672cf62","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.300Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-54eae7ed963686556592ffbb: start","job_status":"start","queue_duration_s":0.231679,"scheduling_latency_s":0.23171}1307gitlab # [ 149.339266] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:40.571Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1308gitlab # [ 149.359248] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:40.590Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"54eae7ed963686556592ffbb","created_at":"2026-09-06T16:28:40.281Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef9f9801542ca796328eb484f672cf62","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.300Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-54eae7ed963686556592ffbb: done: 0.05807 sec","job_status":"done","queue_duration_s":0.231679,"scheduling_latency_s":0.23171,"redis_calls":10,"redis_duration_s":0.012044000000000001,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002468,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007229,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002347,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017884,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002454782,"duration_s":0.05807,"completed_at":"2026-09-06T16:28:40.590Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001988,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1309gitlab # [ 149.424091] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:40.656Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"362d6dd524ab0f9aadbc270f","created_at":"2026-09-06T16:28:40.316Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a6d3c8bef403a5b0a15670ba2a844a9e","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.331Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-362d6dd524ab0f9aadbc270f: start","job_status":"start","queue_duration_s":0.32431,"scheduling_latency_s":0.324346}1310gitlab # [ 149.913794] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:41.145Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"362d6dd524ab0f9aadbc270f","created_at":"2026-09-06T16:28:40.316Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a6d3c8bef403a5b0a15670ba2a844a9e","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.331Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-362d6dd524ab0f9aadbc270f: done: 0.489843 sec","job_status":"done","queue_duration_s":0.32431,"scheduling_latency_s":0.324346,"redis_calls":565,"redis_duration_s":0.24681599999999998,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002436,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.017147,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.227233,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.264745,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000236343,"duration_s":0.489843,"completed_at":"2026-09-06T16:28:41.145Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001991,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1311gitlab # [ 149.959362] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:41.147Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"e33fd68f79fbcc312ac6f2aa","created_at":"2026-09-06T16:28:40.381Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d74cb18075380ade61f5cccebfbe676","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.393Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-e33fd68f79fbcc312ac6f2aa: start","job_status":"start","queue_duration_s":0.753811,"scheduling_latency_s":0.753828}1312gitlab # [ 150.029071] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:41.260Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"e33fd68f79fbcc312ac6f2aa","created_at":"2026-09-06T16:28:40.381Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d74cb18075380ade61f5cccebfbe676","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.393Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-e33fd68f79fbcc312ac6f2aa: done: 0.112999 sec","job_status":"done","queue_duration_s":0.753811,"scheduling_latency_s":0.753828,"redis_calls":12,"redis_duration_s":0.003012,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000514,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001385,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001113,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.089982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000377701,"exclusive_lock_hold_duration_s":0.096458628,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.112999,"completed_at":"2026-09-06T16:28:41.260Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01749,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null}1313gitlab # [ 150.061722] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:41.261Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"ca4fcd088815aa68a861578c","created_at":"2026-09-06T16:28:40.398Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4cd855b3c3b2b518fc2e57aa5ee74482","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.443Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-ca4fcd088815aa68a861578c: start","job_status":"start","queue_duration_s":0.818069,"scheduling_latency_s":0.818087}1314gitlab # [ 150.062667] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:41.278Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"ca4fcd088815aa68a861578c","created_at":"2026-09-06T16:28:40.398Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4cd855b3c3b2b518fc2e57aa5ee74482","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.443Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-ca4fcd088815aa68a861578c: done: 0.016702 sec","job_status":"done","queue_duration_s":0.818069,"scheduling_latency_s":0.818087,"redis_calls":12,"redis_duration_s":0.002572,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000852,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001302,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000418,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00912,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027294,"duration_s":0.016702,"completed_at":"2026-09-06T16:28:41.278Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00112,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1315gitlab # [ 150.101634] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:41.279Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"401a43ac12f3fc62480d0e0d","created_at":"2026-09-06T16:28:40.474Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"634d401e61a445d54882b16f7f54076d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.482Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-401a43ac12f3fc62480d0e0d: start","job_status":"start","queue_duration_s":0.797428,"scheduling_latency_s":0.797442}1316gitlab # [ 150.235274] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:41.467Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"401a43ac12f3fc62480d0e0d","created_at":"2026-09-06T16:28:40.474Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"634d401e61a445d54882b16f7f54076d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.482Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-401a43ac12f3fc62480d0e0d: done: 0.18724 sec","job_status":"done","queue_duration_s":0.797428,"scheduling_latency_s":0.797442,"redis_calls":14,"redis_duration_s":0.003365,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000846,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001271,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001248,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.143169,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000377702,"exclusive_lock_hold_duration_s":0.170206828,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.18724,"completed_at":"2026-09-06T16:28:41.467Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.043775,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1317gitlab # [ 150.303771] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:41.535Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"fb3a90c46b46817360937e64","created_at":"2026-09-06T16:28:40.495Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ccc37253ee2e7831ad39faaefd53029a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:28:40.502Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-fb3a90c46b46817360937e64: start","job_status":"start","queue_duration_s":1.033071,"scheduling_latency_s":1.033106}1318gitlab # [ 150.359751] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:41.591Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"fb3a90c46b46817360937e64","created_at":"2026-09-06T16:28:40.495Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ccc37253ee2e7831ad39faaefd53029a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:28:40.502Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-fb3a90c46b46817360937e64: done: 0.055782 sec","job_status":"done","queue_duration_s":1.033071,"scheduling_latency_s":1.033106,"redis_calls":10,"redis_duration_s":0.002578,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000467,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001581,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00053,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000456762,"duration_s":0.055782,"completed_at":"2026-09-06T16:28:41.591Z","load_balancing_strategy":"primary","db_duration_s":0.005578,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1319gitlab # [ 150.409694] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:41.641Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"98b31ad4116b6f8c3698f6b5","created_at":"2026-09-06T16:28:40.512Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b4e118aac751ee76d31dc170c66fb9b8","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.525Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-98b31ad4116b6f8c3698f6b5: start","job_status":"start","queue_duration_s":1.11565,"scheduling_latency_s":1.115692}1320gitlab # [ 150.447085] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:41.678Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"98b31ad4116b6f8c3698f6b5","created_at":"2026-09-06T16:28:40.512Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b4e118aac751ee76d31dc170c66fb9b8","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.525Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-98b31ad4116b6f8c3698f6b5: done: 0.037911 sec","job_status":"done","queue_duration_s":1.11565,"scheduling_latency_s":1.115692,"redis_calls":10,"redis_duration_s":0.003663,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000641,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002331,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000691,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021403,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000921626,"duration_s":0.037911,"completed_at":"2026-09-06T16:28:41.678Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003623,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1321gitlab # [ 150.488231] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:41.720Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"8da987682d0a30daf6dfdef1","created_at":"2026-09-06T16:28:40.536Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d7c86f2bf736d001c5ac48ef83f3f6cb","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.546Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-8da987682d0a30daf6dfdef1: start","job_status":"start","queue_duration_s":1.173659,"scheduling_latency_s":1.17372}1322gitlab # [ 150.533545] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:41.765Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"8da987682d0a30daf6dfdef1","created_at":"2026-09-06T16:28:40.536Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d7c86f2bf736d001c5ac48ef83f3f6cb","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.546Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-8da987682d0a30daf6dfdef1: done: 0.04544 sec","job_status":"done","queue_duration_s":1.173659,"scheduling_latency_s":1.17372,"redis_calls":10,"redis_duration_s":0.00406,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000743,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002582,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000735,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025302,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000518222,"duration_s":0.04544,"completed_at":"2026-09-06T16:28:41.765Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004203,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1323gitlab # [ 150.573792] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:41.805Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"b3bccaa7ea858e59c8d140b1","created_at":"2026-09-06T16:28:40.563Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5db43f830c4b1e2da9b1cfa51ef9bc0c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.580Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-b3bccaa7ea858e59c8d140b1: start","job_status":"start","queue_duration_s":1.225115,"scheduling_latency_s":1.225178}1324gitlab # [ 150.624462] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:41.856Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"b3bccaa7ea858e59c8d140b1","created_at":"2026-09-06T16:28:40.563Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5db43f830c4b1e2da9b1cfa51ef9bc0c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.580Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-b3bccaa7ea858e59c8d140b1: done: 0.050578 sec","job_status":"done","queue_duration_s":1.225115,"scheduling_latency_s":1.225178,"redis_calls":12,"redis_duration_s":0.003627,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00047,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001779,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001378,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023428,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000771886,"exclusive_lock_hold_duration_s":0.007124928,"duration_s":0.050578,"completed_at":"2026-09-06T16:28:41.856Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002998,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1325gitlab # [ 150.664750] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:41.896Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"d18198a53cc5c23f32cbeb22","created_at":"2026-09-06T16:28:40.653Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f52203024af57ab66ae816ff60cc3170","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.665Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-d18198a53cc5c23f32cbeb22: start","job_status":"start","queue_duration_s":1.230631,"scheduling_latency_s":1.230745}1326gitlab # [ 150.700804] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:41.932Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"d18198a53cc5c23f32cbeb22","created_at":"2026-09-06T16:28:40.653Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f52203024af57ab66ae816ff60cc3170","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.665Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-d18198a53cc5c23f32cbeb22: done: 0.035963 sec","job_status":"done","queue_duration_s":1.230631,"scheduling_latency_s":1.230745,"redis_calls":10,"redis_duration_s":0.003188,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000631,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001938,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000619,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02048,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000499505,"duration_s":0.035963,"completed_at":"2026-09-06T16:28:41.932Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001148,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1327gitlab # [ 150.742778] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:41.974Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"e81a3703e9e12ee82b8cd175","created_at":"2026-09-06T16:28:40.695Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"012194efaddad2bfd0ba51817b2867c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.710Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-e81a3703e9e12ee82b8cd175: start","job_status":"start","queue_duration_s":1.263753,"scheduling_latency_s":1.263805}1328gitlab # [ 150.762631] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:41.994Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1329gitlab # [ 150.775359] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.006Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"e81a3703e9e12ee82b8cd175","created_at":"2026-09-06T16:28:40.695Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"012194efaddad2bfd0ba51817b2867c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.710Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-e81a3703e9e12ee82b8cd175: done: 0.032454 sec","job_status":"done","queue_duration_s":1.263753,"scheduling_latency_s":1.263805,"redis_calls":10,"redis_duration_s":0.003308,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000737,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001925,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000646,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017792,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000498387,"duration_s":0.032454,"completed_at":"2026-09-06T16:28:42.006Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000737,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1330gitlab # [ 150.815809] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.047Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"efb69f8bd5dfddd960f22f72","created_at":"2026-09-06T16:28:40.724Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e87d86fed0bd5501e04656a9c29f5cb","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.737Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-efb69f8bd5dfddd960f22f72: start","job_status":"start","queue_duration_s":1.310487,"scheduling_latency_s":1.310531}1331gitlab # [ 150.875170] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.106Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"efb69f8bd5dfddd960f22f72","created_at":"2026-09-06T16:28:40.724Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e87d86fed0bd5501e04656a9c29f5cb","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.737Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-efb69f8bd5dfddd960f22f72: done: 0.05937 sec","job_status":"done","queue_duration_s":1.310487,"scheduling_latency_s":1.310531,"redis_calls":16,"redis_duration_s":0.004501,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002122,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001787,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000592,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03445,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000481905,"duration_s":0.05937,"completed_at":"2026-09-06T16:28:42.106Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011549,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1332gitlab # [ 150.916575] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.148Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"2716b29312b1848e28e3974a","created_at":"2026-09-06T16:28:40.747Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0fb1bd2993e03a05655e8cf5b8c00b3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.760Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-2716b29312b1848e28e3974a: start","job_status":"start","queue_duration_s":1.388295,"scheduling_latency_s":1.388333}1333gitlab # [ 150.984108] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.215Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"2716b29312b1848e28e3974a","created_at":"2026-09-06T16:28:40.747Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0fb1bd2993e03a05655e8cf5b8c00b3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.760Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-2716b29312b1848e28e3974a: done: 0.067638 sec","job_status":"done","queue_duration_s":1.388295,"scheduling_latency_s":1.388333,"redis_calls":14,"redis_duration_s":0.0041010000000000005,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001103,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001754,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001244,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030303,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000720762,"exclusive_lock_hold_duration_s":0.026581033,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.067638,"completed_at":"2026-09-06T16:28:42.215Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015805,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1334gitlab # [ 151.046576] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.277Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"0276b0a4559c6546edfeeb91","created_at":"2026-09-06T16:28:40.791Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"86e92d11fb60fa5f70f6699256ef065d","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.801Z","job_size_bytes":3,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-0276b0a4559c6546edfeeb91: start","job_status":"start","queue_duration_s":1.476639,"scheduling_latency_s":1.47668}1335gitlab # [ 151.077525] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.309Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"0276b0a4559c6546edfeeb91","created_at":"2026-09-06T16:28:40.791Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"86e92d11fb60fa5f70f6699256ef065d","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.801Z","job_size_bytes":3,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-0276b0a4559c6546edfeeb91: done: 0.032036 sec","job_status":"done","queue_duration_s":1.476639,"scheduling_latency_s":1.47668,"redis_calls":10,"redis_duration_s":0.0016640000000000001,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000362,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000955,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000347,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006408,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000260927,"duration_s":0.032036,"completed_at":"2026-09-06T16:28:42.309Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0009,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1336gitlab # [ 151.118050] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.349Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"1273d17939da939018808bb6","created_at":"2026-09-06T16:28:40.810Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f847b98662c87d8f481611d8b87bb0b2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.817Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-1273d17939da939018808bb6: start","job_status":"start","queue_duration_s":1.531932,"scheduling_latency_s":1.531992}1337gitlab # [ 151.169568] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.401Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"1273d17939da939018808bb6","created_at":"2026-09-06T16:28:40.810Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f847b98662c87d8f481611d8b87bb0b2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.817Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-1273d17939da939018808bb6: done: 0.05185 sec","job_status":"done","queue_duration_s":1.531932,"scheduling_latency_s":1.531992,"redis_calls":19,"redis_duration_s":0.005833,"redis_read_bytes":22,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001225,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000341,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003616,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000651,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03465,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000417092,"duration_s":0.05185,"completed_at":"2026-09-06T16:28:42.401Z","load_balancing_strategy":"primary","db_duration_s":0.002066,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1338gitlab # [ 151.211167] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.443Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"747b911b9349251b582870dc","created_at":"2026-09-06T16:28:40.826Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"01ec513130edfd3680b0665901a3f965","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.833Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-747b911b9349251b582870dc: start","job_status":"start","queue_duration_s":1.609069,"scheduling_latency_s":1.609127}1339gitlab # [ 151.243466] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.475Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"747b911b9349251b582870dc","created_at":"2026-09-06T16:28:40.826Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"01ec513130edfd3680b0665901a3f965","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.833Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-747b911b9349251b582870dc: done: 0.032349 sec","job_status":"done","queue_duration_s":1.609069,"scheduling_latency_s":1.609127,"redis_calls":12,"redis_duration_s":0.004099,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000904,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000629,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001916,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00065,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00047073,"duration_s":0.032349,"completed_at":"2026-09-06T16:28:42.475Z","load_balancing_strategy":"primary","db_duration_s":0.000793,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1340gitlab # [ 151.283947] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.515Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"5b8e1715fc53e92388e72df6","created_at":"2026-09-06T16:28:40.841Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"837c566e6fc5d8d8a04e9fe9bcff8f09","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.848Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-5b8e1715fc53e92388e72df6: start","job_status":"start","queue_duration_s":1.66743,"scheduling_latency_s":1.667476}1341gitlab # [ 151.325541] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.557Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"5b8e1715fc53e92388e72df6","created_at":"2026-09-06T16:28:40.841Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"837c566e6fc5d8d8a04e9fe9bcff8f09","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.848Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-5b8e1715fc53e92388e72df6: done: 0.041203 sec","job_status":"done","queue_duration_s":1.66743,"scheduling_latency_s":1.667476,"redis_calls":12,"redis_duration_s":0.003493,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000626,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0017,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001167,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000695061,"exclusive_lock_hold_duration_s":0.009079087,"duration_s":0.041203,"completed_at":"2026-09-06T16:28:42.556Z","load_balancing_strategy":"primary","db_duration_s":0.00428,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1342gitlab # [ 151.365708] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.597Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ff8a5da25bd113835d77254d","created_at":"2026-09-06T16:28:40.856Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.862Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ff8a5da25bd113835d77254d: start","job_status":"start","queue_duration_s":1.734716,"scheduling_latency_s":1.734823}1343gitlab # [ 151.504341] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.736Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ff8a5da25bd113835d77254d","created_at":"2026-09-06T16:28:40.856Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.862Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ff8a5da25bd113835d77254d: done: 0.138836 sec","job_status":"done","queue_duration_s":1.734716,"scheduling_latency_s":1.734823,"redis_calls":71,"redis_duration_s":0.020199,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002483,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001211,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.015553,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000952,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.11371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000428267,"duration_s":0.138836,"completed_at":"2026-09-06T16:28:42.736Z","load_balancing_strategy":"primary","db_duration_s":0.002736,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1344gitlab # [ 151.578485] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.737Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"22828b7dfe05ff83ad1a85ca","created_at":"2026-09-06T16:28:40.868Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b4bf144c780b3300b30a5442f9be507a","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.874Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-22828b7dfe05ff83ad1a85ca: start","job_status":"start","queue_duration_s":1.863081,"scheduling_latency_s":1.863098}1345gitlab # [ 151.591969] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.761Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"22828b7dfe05ff83ad1a85ca","created_at":"2026-09-06T16:28:40.868Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b4bf144c780b3300b30a5442f9be507a","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.874Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-22828b7dfe05ff83ad1a85ca: done: 0.024012 sec","job_status":"done","queue_duration_s":1.863081,"scheduling_latency_s":1.863098,"redis_calls":13,"redis_duration_s":0.003065,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000478,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001605,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000982,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013974,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.007260421,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.024012,"completed_at":"2026-09-06T16:28:42.761Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003268,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1346gitlab # [ 151.632741] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.763Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"f5b39f29c09184976ab93fa5","created_at":"2026-09-06T16:28:40.884Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bdca11ece490f0ca643ae5ca74d69adc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-06T16:28:40.889Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-f5b39f29c09184976ab93fa5: start","job_status":"start","queue_duration_s":1.873696,"scheduling_latency_s":1.873711}1347gitlab # [ 151.645451] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.771Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1348gitlab # [ 151.646132] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.776Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"f5b39f29c09184976ab93fa5","created_at":"2026-09-06T16:28:40.884Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bdca11ece490f0ca643ae5ca74d69adc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-06T16:28:40.889Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-f5b39f29c09184976ab93fa5: done: 0.01313 sec","job_status":"done","queue_duration_s":1.873696,"scheduling_latency_s":1.873711,"redis_calls":10,"redis_duration_s":0.002094,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000481,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001147,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000466,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000273778,"duration_s":0.01313,"completed_at":"2026-09-06T16:28:42.776Z","load_balancing_strategy":"primary","db_duration_s":0.000408,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1349gitlab # [ 151.647069] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.777Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"f18ba36acbdd24288c5510cb","created_at":"2026-09-06T16:28:40.894Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"25dfec7601149323023e7d18b49dbed5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.899Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-f18ba36acbdd24288c5510cb: start","job_status":"start","queue_duration_s":1.877631,"scheduling_latency_s":1.877645}1350gitlab # [ 151.658900] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.815Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"f18ba36acbdd24288c5510cb","created_at":"2026-09-06T16:28:40.894Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"25dfec7601149323023e7d18b49dbed5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.899Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-f18ba36acbdd24288c5510cb: done: 0.037945 sec","job_status":"done","queue_duration_s":1.877631,"scheduling_latency_s":1.877645,"redis_calls":12,"redis_duration_s":0.003091,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000637,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001375,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001079,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024579,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00140828,"exclusive_lock_hold_duration_s":0.011446985,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.037945,"completed_at":"2026-09-06T16:28:42.815Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007529,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1351gitlab # [ 151.736103] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.816Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"87e53e8f770f8f3b4b00f508","created_at":"2026-09-06T16:28:40.907Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c5697bb2d24a9b68f4217937d5c875e4","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.911Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-87e53e8f770f8f3b4b00f508: start","job_status":"start","queue_duration_s":1.904763,"scheduling_latency_s":1.904778}1352gitlab # [ 151.736778] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.834Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"87e53e8f770f8f3b4b00f508","created_at":"2026-09-06T16:28:40.907Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c5697bb2d24a9b68f4217937d5c875e4","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.911Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-87e53e8f770f8f3b4b00f508: done: 0.01764 sec","job_status":"done","queue_duration_s":1.904763,"scheduling_latency_s":1.904778,"redis_calls":12,"redis_duration_s":0.0026950000000000003,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000438,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00126,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000997,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009135,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000468216,"exclusive_lock_hold_duration_s":0.003019658,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.01764,"completed_at":"2026-09-06T16:28:42.834Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001902,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1353gitlab # [ 151.776628] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.835Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"654cbb8ba9d59f97202846fa","created_at":"2026-09-06T16:28:40.918Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9cd794fb64d3e13606b7bfbd7cbcc757","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.923Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-654cbb8ba9d59f97202846fa: start","job_status":"start","queue_duration_s":1.911829,"scheduling_latency_s":1.911843}1354gitlab # [ 151.777431] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.843Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1355gitlab # [ 151.778065] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.856Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"654cbb8ba9d59f97202846fa","created_at":"2026-09-06T16:28:40.918Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9cd794fb64d3e13606b7bfbd7cbcc757","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-06T16:28:40.923Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-654cbb8ba9d59f97202846fa: done: 0.021444 sec","job_status":"done","queue_duration_s":1.911829,"scheduling_latency_s":1.911843,"redis_calls":10,"redis_duration_s":0.00356,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000446,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002665,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000449,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010053,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000459277,"duration_s":0.021444,"completed_at":"2026-09-06T16:28:42.856Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000479,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1356gitlab # [ 151.821916] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:42.859Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4017fa6fa4e71df4ee2b3520","created_at":"2026-09-06T16:28:42.374Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-6ef91aa04de842ae","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f847b98662c87d8f481611d8b87bb0b2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.391Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4017fa6fa4e71df4ee2b3520: start","job_status":"start","queue_duration_s":0.467583,"scheduling_latency_s":0.467618}1357gitlab # [ 151.861153] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.093Z","message":"No existing merge request to be cleaned up."}1358gitlab # [ 151.884675] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.116Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4017fa6fa4e71df4ee2b3520","created_at":"2026-09-06T16:28:42.374Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-6ef91aa04de842ae","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f847b98662c87d8f481611d8b87bb0b2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.391Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4017fa6fa4e71df4ee2b3520: done: 0.257039 sec","job_status":"done","queue_duration_s":0.467583,"scheduling_latency_s":0.467618,"redis_calls":13,"redis_duration_s":0.004986,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000359,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001165,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003462,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.214,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.215,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.215,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.214,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028451,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000424076,"duration_s":0.257039,"completed_at":"2026-09-06T16:28:43.116Z","load_balancing_strategy":"primary","db_duration_s":0.216396,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1359gitlab # [ 151.923510] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.155Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"42d429759d09a7301ab66383","created_at":"2026-09-06T16:28:42.374Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-6ef91aa04de842ae","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f847b98662c87d8f481611d8b87bb0b2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.391Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-42d429759d09a7301ab66383: start","job_status":"start","queue_duration_s":0.763559,"scheduling_latency_s":0.763605}1360gitlab # [ 151.950697] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.182Z","message":"No existing merge request to be cleaned up."}1361gitlab # [ 151.965722] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.197Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"42d429759d09a7301ab66383","created_at":"2026-09-06T16:28:42.374Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-6ef91aa04de842ae","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f847b98662c87d8f481611d8b87bb0b2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.391Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-42d429759d09a7301ab66383: done: 0.041904 sec","job_status":"done","queue_duration_s":0.763559,"scheduling_latency_s":0.763605,"redis_calls":10,"redis_duration_s":0.004261,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001079,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003182,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020834,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041904,"completed_at":"2026-09-06T16:28:43.197Z","load_balancing_strategy":"primary","db_duration_s":0.003231,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1362gitlab # [ 152.002684] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.234Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"59a8a9c8cbc631885b841552","created_at":"2026-09-06T16:28:42.374Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-6ef91aa04de842ae","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f847b98662c87d8f481611d8b87bb0b2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.391Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-59a8a9c8cbc631885b841552: start","job_status":"start","queue_duration_s":0.842673,"scheduling_latency_s":0.842779}1363gitlab # [ 152.029720] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.261Z","message":"No existing merge request to be cleaned up."}1364gitlab # [ 152.044554] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.275Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"59a8a9c8cbc631885b841552","created_at":"2026-09-06T16:28:42.374Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-6ef91aa04de842ae","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f847b98662c87d8f481611d8b87bb0b2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.391Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-59a8a9c8cbc631885b841552: done: 0.041595 sec","job_status":"done","queue_duration_s":0.842673,"scheduling_latency_s":0.842779,"redis_calls":10,"redis_duration_s":0.0046170000000000004,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000969,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003648,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02066,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041595,"completed_at":"2026-09-06T16:28:43.275Z","load_balancing_strategy":"primary","db_duration_s":0.003276,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1365gitlab # [ 152.084128] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.315Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c8efdc8a256e77f4248f2cde","created_at":"2026-09-06T16:28:42.374Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-6ef91aa04de842ae","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f847b98662c87d8f481611d8b87bb0b2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.391Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c8efdc8a256e77f4248f2cde: start","job_status":"start","queue_duration_s":0.923623,"scheduling_latency_s":0.923667}1366gitlab # [ 152.115716] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.347Z","message":"No existing merge request to be cleaned up."}1367gitlab # [ 152.129437] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.361Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c8efdc8a256e77f4248f2cde","created_at":"2026-09-06T16:28:42.374Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-6ef91aa04de842ae","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f847b98662c87d8f481611d8b87bb0b2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.391Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c8efdc8a256e77f4248f2cde: done: 0.045996 sec","job_status":"done","queue_duration_s":0.923623,"scheduling_latency_s":0.923667,"redis_calls":10,"redis_duration_s":0.004251,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000968,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003283,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020639,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.045996,"completed_at":"2026-09-06T16:28:43.361Z","load_balancing_strategy":"primary","db_duration_s":0.003059,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1368gitlab # [ 152.165780] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"165a16ff5fe4308c81253f7f","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-165a16ff5fe4308c81253f7f: start","job_status":"start","queue_duration_s":0.672895,"scheduling_latency_s":0.672936}1369gitlab # [ 152.194556] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.426Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"165a16ff5fe4308c81253f7f","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-165a16ff5fe4308c81253f7f: done: 0.028649 sec","job_status":"done","queue_duration_s":0.672895,"scheduling_latency_s":0.672936,"redis_calls":16,"redis_duration_s":0.0043549999999999995,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000549,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001294,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002512,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000486933,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028649,"completed_at":"2026-09-06T16:28:43.426Z","load_balancing_strategy":"primary","db_duration_s":0.000677,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1370gitlab # [ 152.236609] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.468Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"475a30f6b4f43475bb021d1e","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-475a30f6b4f43475bb021d1e: start","job_status":"start","queue_duration_s":0.743534,"scheduling_latency_s":0.743596}1371gitlab # [ 152.257488] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.489Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"475a30f6b4f43475bb021d1e","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-475a30f6b4f43475bb021d1e: done: 0.020906 sec","job_status":"done","queue_duration_s":0.743534,"scheduling_latency_s":0.743596,"redis_calls":13,"redis_duration_s":0.00334,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001035,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002305,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020906,"completed_at":"2026-09-06T16:28:43.489Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1372gitlab # [ 152.294852] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.526Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cf48f907a295098bfa45adc","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cf48f907a295098bfa45adc: start","job_status":"start","queue_duration_s":0.80191,"scheduling_latency_s":0.80195}1373gitlab # [ 152.335696] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.566Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cf48f907a295098bfa45adc","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cf48f907a295098bfa45adc: done: 0.040021 sec","job_status":"done","queue_duration_s":0.80191,"scheduling_latency_s":0.80195,"redis_calls":13,"redis_duration_s":0.003842,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001351,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002491,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.040021,"completed_at":"2026-09-06T16:28:43.566Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1374gitlab # [ 152.373358] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.605Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7bd345769071b3367ccd90a","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7bd345769071b3367ccd90a: start","job_status":"start","queue_duration_s":0.88013,"scheduling_latency_s":0.880176}1375gitlab # [ 152.408130] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.639Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7bd345769071b3367ccd90a","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7bd345769071b3367ccd90a: done: 0.034388 sec","job_status":"done","queue_duration_s":0.88013,"scheduling_latency_s":0.880176,"redis_calls":13,"redis_duration_s":0.002924,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001075,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001849,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016893,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034388,"completed_at":"2026-09-06T16:28:43.639Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1376gitlab # [ 152.446534] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8a9960713351656c094fe5f","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8a9960713351656c094fe5f: start","job_status":"start","queue_duration_s":0.953436,"scheduling_latency_s":0.953479}1377gitlab # [ 152.485789] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.717Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8a9960713351656c094fe5f","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8a9960713351656c094fe5f: done: 0.038938 sec","job_status":"done","queue_duration_s":0.953436,"scheduling_latency_s":0.953479,"redis_calls":13,"redis_duration_s":0.004045,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001337,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002708,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016965,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038938,"completed_at":"2026-09-06T16:28:43.717Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1378gitlab # [ 152.508105] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.740Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6e314af0b82b9067ceebe3a","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6e314af0b82b9067ceebe3a: start","job_status":"start","queue_duration_s":1.015125,"scheduling_latency_s":1.015167}1379gitlab # [ 152.533572] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6e314af0b82b9067ceebe3a","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6e314af0b82b9067ceebe3a: done: 0.025196 sec","job_status":"done","queue_duration_s":1.015125,"scheduling_latency_s":1.015167,"redis_calls":13,"redis_duration_s":0.003958,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001248,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00271,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016542,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025196,"completed_at":"2026-09-06T16:28:43.765Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1380gitlab # [ 152.555720] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.767Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"636bd892368287f949acdf48","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-636bd892368287f949acdf48: start","job_status":"start","queue_duration_s":1.042648,"scheduling_latency_s":1.042676}1381gitlab # [ 152.605456] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.837Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"636bd892368287f949acdf48","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-636bd892368287f949acdf48: done: 0.069801 sec","job_status":"done","queue_duration_s":1.042648,"scheduling_latency_s":1.042676,"redis_calls":13,"redis_duration_s":0.003156,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002151,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.060567,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.069801,"completed_at":"2026-09-06T16:28:43.837Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1382gitlab # [ 152.643205] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.874Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a7a41ad3460eafffd173c04","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a7a41ad3460eafffd173c04: start","job_status":"start","queue_duration_s":1.149931,"scheduling_latency_s":1.149987}1383gitlab # [ 152.671266] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.903Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a7a41ad3460eafffd173c04","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a7a41ad3460eafffd173c04: done: 0.028264 sec","job_status":"done","queue_duration_s":1.149931,"scheduling_latency_s":1.149987,"redis_calls":13,"redis_duration_s":0.0037070000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001396,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002311,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018405,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028264,"completed_at":"2026-09-06T16:28:43.902Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1384gitlab # [ 152.709315] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.941Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7673f5fa1caabf794f417a0d","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7673f5fa1caabf794f417a0d: start","job_status":"start","queue_duration_s":1.216244,"scheduling_latency_s":1.216289}1385gitlab # [ 152.734339] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:43.965Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7673f5fa1caabf794f417a0d","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7673f5fa1caabf794f417a0d: done: 0.024353 sec","job_status":"done","queue_duration_s":1.216244,"scheduling_latency_s":1.216289,"redis_calls":13,"redis_duration_s":0.0037689999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001247,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002522,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024353,"completed_at":"2026-09-06T16:28:43.965Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1386gitlab # [ 152.772081] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.004Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e7fb5ca468b4667cc8451dd","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e7fb5ca468b4667cc8451dd: start","job_status":"start","queue_duration_s":1.279057,"scheduling_latency_s":1.279104}1387gitlab # [ 152.797104] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.028Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e7fb5ca468b4667cc8451dd","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e7fb5ca468b4667cc8451dd: done: 0.025 sec","job_status":"done","queue_duration_s":1.279057,"scheduling_latency_s":1.279104,"redis_calls":13,"redis_duration_s":0.003803,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001273,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00253,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025,"completed_at":"2026-09-06T16:28:44.028Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1388gitlab # [ 152.833921] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.065Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9e6bfe725940b4e05c6bd55","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9e6bfe725940b4e05c6bd55: start","job_status":"start","queue_duration_s":1.340842,"scheduling_latency_s":1.340886}1389gitlab # [ 152.867155] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.098Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9e6bfe725940b4e05c6bd55","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9e6bfe725940b4e05c6bd55: done: 0.033192 sec","job_status":"done","queue_duration_s":1.340842,"scheduling_latency_s":1.340886,"redis_calls":13,"redis_duration_s":0.0029500000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000982,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015995,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033192,"completed_at":"2026-09-06T16:28:44.098Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1390gitlab # [ 152.904607] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.136Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2db455f23e0b66086eac55f3","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2db455f23e0b66086eac55f3: start","job_status":"start","queue_duration_s":1.411437,"scheduling_latency_s":1.411482}1391gitlab # [ 152.929453] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.161Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2db455f23e0b66086eac55f3","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2db455f23e0b66086eac55f3: done: 0.024943 sec","job_status":"done","queue_duration_s":1.411437,"scheduling_latency_s":1.411482,"redis_calls":13,"redis_duration_s":0.003719,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00124,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002479,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024943,"completed_at":"2026-09-06T16:28:44.161Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1392gitlab # [ 152.967155] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.198Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f5ed159f10881a096434640","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f5ed159f10881a096434640: start","job_status":"start","queue_duration_s":1.473373,"scheduling_latency_s":1.473421}1393gitlab # [ 152.999860] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.231Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f5ed159f10881a096434640","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f5ed159f10881a096434640: done: 0.033296 sec","job_status":"done","queue_duration_s":1.473373,"scheduling_latency_s":1.473421,"redis_calls":13,"redis_duration_s":0.002752,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000925,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001827,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033296,"completed_at":"2026-09-06T16:28:44.231Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1394gitlab # [ 153.036797] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.268Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2caa0f9e7bbe0869ac10250e","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2caa0f9e7bbe0869ac10250e: start","job_status":"start","queue_duration_s":1.543787,"scheduling_latency_s":1.543839}1395gitlab # [ 153.061633] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.293Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2caa0f9e7bbe0869ac10250e","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2caa0f9e7bbe0869ac10250e: done: 0.024765 sec","job_status":"done","queue_duration_s":1.543787,"scheduling_latency_s":1.543839,"redis_calls":13,"redis_duration_s":0.003713,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001315,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002398,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01571,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024765,"completed_at":"2026-09-06T16:28:44.293Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1396gitlab # [ 153.099448] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.330Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae05e0a58902d5ec4da8db13","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae05e0a58902d5ec4da8db13: start","job_status":"start","queue_duration_s":1.605639,"scheduling_latency_s":1.605684}1397gitlab # [ 153.123797] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.355Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae05e0a58902d5ec4da8db13","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae05e0a58902d5ec4da8db13: done: 0.025284 sec","job_status":"done","queue_duration_s":1.605639,"scheduling_latency_s":1.605684,"redis_calls":13,"redis_duration_s":0.0037660000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001399,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002367,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025284,"completed_at":"2026-09-06T16:28:44.355Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1398gitlab # [ 153.161116] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.392Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d071cf4ee2faabd0e6298e61","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d071cf4ee2faabd0e6298e61: start","job_status":"start","queue_duration_s":1.66794,"scheduling_latency_s":1.667984}1399gitlab # [ 153.191141] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.423Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d071cf4ee2faabd0e6298e61","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d071cf4ee2faabd0e6298e61: done: 0.030296 sec","job_status":"done","queue_duration_s":1.66794,"scheduling_latency_s":1.667984,"redis_calls":13,"redis_duration_s":0.002575,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001653,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013195,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030296,"completed_at":"2026-09-06T16:28:44.423Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1400gitlab # [ 153.228666] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.460Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64d2df17cbe138464e375a07","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64d2df17cbe138464e375a07: start","job_status":"start","queue_duration_s":1.735539,"scheduling_latency_s":1.735606}1401gitlab # [ 153.265895] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.497Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64d2df17cbe138464e375a07","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64d2df17cbe138464e375a07: done: 0.037664 sec","job_status":"done","queue_duration_s":1.735539,"scheduling_latency_s":1.735606,"redis_calls":13,"redis_duration_s":0.003111,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001198,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016383,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.037664,"completed_at":"2026-09-06T16:28:44.497Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1402gitlab # [ 153.305173] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.537Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df01758f766d70739917f224","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df01758f766d70739917f224: start","job_status":"start","queue_duration_s":1.812078,"scheduling_latency_s":1.812126}1403gitlab # [ 153.338390] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.570Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df01758f766d70739917f224","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df01758f766d70739917f224: done: 0.033133 sec","job_status":"done","queue_duration_s":1.812078,"scheduling_latency_s":1.812126,"redis_calls":13,"redis_duration_s":0.002891,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001882,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016005,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033133,"completed_at":"2026-09-06T16:28:44.570Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1404gitlab # [ 153.376315] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.608Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e8e60476d6c0f762f52840c8","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e8e60476d6c0f762f52840c8: start","job_status":"start","queue_duration_s":1.883205,"scheduling_latency_s":1.883252}1405gitlab # [ 153.401424] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.633Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e8e60476d6c0f762f52840c8","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e8e60476d6c0f762f52840c8: done: 0.025151 sec","job_status":"done","queue_duration_s":1.883205,"scheduling_latency_s":1.883252,"redis_calls":13,"redis_duration_s":0.003895,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001334,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002561,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015826,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025151,"completed_at":"2026-09-06T16:28:44.633Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1406gitlab # [ 153.439606] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.670Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f66e2b1cd4ab938a6250f0e","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f66e2b1cd4ab938a6250f0e: start","job_status":"start","queue_duration_s":1.945915,"scheduling_latency_s":1.945959}1407gitlab # [ 153.464378] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.696Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f66e2b1cd4ab938a6250f0e","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f66e2b1cd4ab938a6250f0e: done: 0.025385 sec","job_status":"done","queue_duration_s":1.945915,"scheduling_latency_s":1.945959,"redis_calls":13,"redis_duration_s":0.0037199999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0013,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00242,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025385,"completed_at":"2026-09-06T16:28:44.696Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1408gitlab # [ 153.501930] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.733Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cd4ad7173aad1f9368871a44","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cd4ad7173aad1f9368871a44: start","job_status":"start","queue_duration_s":2.008846,"scheduling_latency_s":2.00889}1409gitlab # [ 153.535334] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.767Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cd4ad7173aad1f9368871a44","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cd4ad7173aad1f9368871a44: done: 0.033307 sec","job_status":"done","queue_duration_s":2.008846,"scheduling_latency_s":2.00889,"redis_calls":13,"redis_duration_s":0.002905,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000976,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001929,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033307,"completed_at":"2026-09-06T16:28:44.766Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410gitlab # [ 153.572698] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.804Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f1630f1668dca3b2567870a","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f1630f1668dca3b2567870a: start","job_status":"start","queue_duration_s":2.079687,"scheduling_latency_s":2.079733}1411gitlab # [ 153.597257] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.829Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f1630f1668dca3b2567870a","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f1630f1668dca3b2567870a: done: 0.02456 sec","job_status":"done","queue_duration_s":2.079687,"scheduling_latency_s":2.079733,"redis_calls":13,"redis_duration_s":0.004071,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001656,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002415,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015645,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02456,"completed_at":"2026-09-06T16:28:44.829Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412gitlab # [ 153.634727] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6ab7ba11713d9032237e930","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6ab7ba11713d9032237e930: start","job_status":"start","queue_duration_s":2.141722,"scheduling_latency_s":2.141768}1413gitlab # [ 153.669818] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.901Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6ab7ba11713d9032237e930","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6ab7ba11713d9032237e930: done: 0.035112 sec","job_status":"done","queue_duration_s":2.141722,"scheduling_latency_s":2.141768,"redis_calls":13,"redis_duration_s":0.003893,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001399,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002494,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035112,"completed_at":"2026-09-06T16:28:44.901Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1414gitlab # [ 153.709617] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.941Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e794426d0302eb7cd987db48","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e794426d0302eb7cd987db48: start","job_status":"start","queue_duration_s":2.216304,"scheduling_latency_s":2.216354}1415gitlab # [ 153.734642] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:44.966Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e794426d0302eb7cd987db48","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e794426d0302eb7cd987db48: done: 0.026063 sec","job_status":"done","queue_duration_s":2.216304,"scheduling_latency_s":2.216354,"redis_calls":13,"redis_duration_s":0.0037539999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001158,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002596,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015153,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026063,"completed_at":"2026-09-06T16:28:44.966Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416gitlab # [ 153.771922] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.003Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b79bb7a76f6f821721dc2122","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b79bb7a76f6f821721dc2122: start","job_status":"start","queue_duration_s":2.278948,"scheduling_latency_s":2.278993}1417gitlab # [ 153.807275] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.039Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b79bb7a76f6f821721dc2122","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b79bb7a76f6f821721dc2122: done: 0.035217 sec","job_status":"done","queue_duration_s":2.278948,"scheduling_latency_s":2.278993,"redis_calls":13,"redis_duration_s":0.0046370000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001022,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003615,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035217,"completed_at":"2026-09-06T16:28:45.038Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1418gitlab # [ 153.844872] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.076Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22577f4fce79884898aefb24","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22577f4fce79884898aefb24: start","job_status":"start","queue_duration_s":2.351884,"scheduling_latency_s":2.351928}1419gitlab # [ 153.870217] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.102Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22577f4fce79884898aefb24","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22577f4fce79884898aefb24: done: 0.025282 sec","job_status":"done","queue_duration_s":2.351884,"scheduling_latency_s":2.351928,"redis_calls":13,"redis_duration_s":0.0038,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001287,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002513,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025282,"completed_at":"2026-09-06T16:28:45.101Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1420gitlab # [ 153.907715] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.139Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f847795c2960e3d928a9aee3","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f847795c2960e3d928a9aee3: start","job_status":"start","queue_duration_s":2.414698,"scheduling_latency_s":2.414744}1421gitlab # [ 153.932534] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.164Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f847795c2960e3d928a9aee3","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f847795c2960e3d928a9aee3: done: 0.024922 sec","job_status":"done","queue_duration_s":2.414698,"scheduling_latency_s":2.414744,"redis_calls":13,"redis_duration_s":0.0037089999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001286,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002423,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024922,"completed_at":"2026-09-06T16:28:45.164Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1422gitlab # [ 154.072178] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.303Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9192cd7b2b0c87eab2756e2a","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9192cd7b2b0c87eab2756e2a: start","job_status":"start","queue_duration_s":2.578063,"scheduling_latency_s":2.578102}1423gitlab # [ 154.122950] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.354Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9192cd7b2b0c87eab2756e2a","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9192cd7b2b0c87eab2756e2a: done: 0.051067 sec","job_status":"done","queue_duration_s":2.578063,"scheduling_latency_s":2.578102,"redis_calls":13,"redis_duration_s":0.003955,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001293,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002662,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016304,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.051067,"completed_at":"2026-09-06T16:28:45.353Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1424gitlab # [ 154.169356] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.356Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce9a3eec96c60369545c3bdf","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce9a3eec96c60369545c3bdf: start","job_status":"start","queue_duration_s":2.631311,"scheduling_latency_s":2.631344}1425gitlab # [ 154.171218] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce9a3eec96c60369545c3bdf","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce9a3eec96c60369545c3bdf: done: 0.020606 sec","job_status":"done","queue_duration_s":2.631311,"scheduling_latency_s":2.631344,"redis_calls":13,"redis_duration_s":0.003668,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0013,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002368,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012699,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020606,"completed_at":"2026-09-06T16:28:45.376Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1426gitlab # [ 154.191193] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1fc4c81f808069308eade89","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1fc4c81f808069308eade89: start","job_status":"start","queue_duration_s":2.653543,"scheduling_latency_s":2.653624}1427gitlab # [ 154.235242] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.388Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1fc4c81f808069308eade89","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1fc4c81f808069308eade89: done: 0.00976 sec","job_status":"done","queue_duration_s":2.653543,"scheduling_latency_s":2.653624,"redis_calls":13,"redis_duration_s":0.001648,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000601,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001047,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006019,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00976,"completed_at":"2026-09-06T16:28:45.388Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1428gitlab # [ 154.237945] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.389Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef87a9193cb82d880c0835fa","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef87a9193cb82d880c0835fa: start","job_status":"start","queue_duration_s":2.664138,"scheduling_latency_s":2.664152}1429gitlab # [ 154.284582] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef87a9193cb82d880c0835fa","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef87a9193cb82d880c0835fa: done: 0.00931 sec","job_status":"done","queue_duration_s":2.664138,"scheduling_latency_s":2.664152,"redis_calls":13,"redis_duration_s":0.0015760000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001043,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005906,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00931,"completed_at":"2026-09-06T16:28:45.398Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1430gitlab # [ 154.285640] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.399Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d23616b4d562888a882182c","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d23616b4d562888a882182c: start","job_status":"start","queue_duration_s":2.674432,"scheduling_latency_s":2.674445}1431gitlab # [ 154.332898] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.412Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d23616b4d562888a882182c","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d23616b4d562888a882182c: done: 0.013036 sec","job_status":"done","queue_duration_s":2.674432,"scheduling_latency_s":2.674445,"redis_calls":13,"redis_duration_s":0.005042,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00088,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004162,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013036,"completed_at":"2026-09-06T16:28:45.412Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1432gitlab # [ 154.333943] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.413Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43bbefcc37ab31659ea82185","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43bbefcc37ab31659ea82185: start","job_status":"start","queue_duration_s":2.688416,"scheduling_latency_s":2.68843}1433gitlab # [ 154.375036] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.422Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43bbefcc37ab31659ea82185","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43bbefcc37ab31659ea82185: done: 0.00949 sec","job_status":"done","queue_duration_s":2.688416,"scheduling_latency_s":2.68843,"redis_calls":13,"redis_duration_s":0.001727,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001194,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00606,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00949,"completed_at":"2026-09-06T16:28:45.422Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1434gitlab # [ 154.378704] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.423Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf81820bc069f3866e188e3c","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf81820bc069f3866e188e3c: start","job_status":"start","queue_duration_s":2.698766,"scheduling_latency_s":2.69878}1435gitlab # [ 154.420086] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.433Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf81820bc069f3866e188e3c","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf81820bc069f3866e188e3c: done: 0.009712 sec","job_status":"done","queue_duration_s":2.698766,"scheduling_latency_s":2.69878,"redis_calls":13,"redis_duration_s":0.001594,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001069,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009712,"completed_at":"2026-09-06T16:28:45.433Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1436gitlab # [ 154.424255] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.434Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cca366f99a2881f6738aade0","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cca366f99a2881f6738aade0: start","job_status":"start","queue_duration_s":2.709416,"scheduling_latency_s":2.70943}1437gitlab # [ 154.464963] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.443Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cca366f99a2881f6738aade0","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cca366f99a2881f6738aade0: done: 0.009395 sec","job_status":"done","queue_duration_s":2.709416,"scheduling_latency_s":2.70943,"redis_calls":13,"redis_duration_s":0.001653,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000573,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00108,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005864,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009395,"completed_at":"2026-09-06T16:28:45.443Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1438gitlab # [ 154.469807] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.444Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b2c11a46c2eb8926c121ce6","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b2c11a46c2eb8926c121ce6: start","job_status":"start","queue_duration_s":2.719896,"scheduling_latency_s":2.71991}1439gitlab # [ 154.507749] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.454Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b2c11a46c2eb8926c121ce6","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b2c11a46c2eb8926c121ce6: done: 0.009363 sec","job_status":"done","queue_duration_s":2.719896,"scheduling_latency_s":2.71991,"redis_calls":13,"redis_duration_s":0.001727,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001206,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009363,"completed_at":"2026-09-06T16:28:45.454Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1440gitlab # [ 154.515451] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.455Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a69e1506940a98a22d5454f9","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a69e1506940a98a22d5454f9: start","job_status":"start","queue_duration_s":2.730117,"scheduling_latency_s":2.730132}1441gitlab # [ 154.526714] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.464Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a69e1506940a98a22d5454f9","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a69e1506940a98a22d5454f9: done: 0.009279 sec","job_status":"done","queue_duration_s":2.730117,"scheduling_latency_s":2.730132,"redis_calls":13,"redis_duration_s":0.001604,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001064,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009279,"completed_at":"2026-09-06T16:28:45.464Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1442gitlab # [ 154.574181] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.465Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f31768efcbf4d7ebdc103d88","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f31768efcbf4d7ebdc103d88: start","job_status":"start","queue_duration_s":2.740269,"scheduling_latency_s":2.740284}1443gitlab # [ 154.574877] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.474Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f31768efcbf4d7ebdc103d88","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f31768efcbf4d7ebdc103d88: done: 0.00927 sec","job_status":"done","queue_duration_s":2.740269,"scheduling_latency_s":2.740284,"redis_calls":13,"redis_duration_s":0.0016120000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001103,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00927,"completed_at":"2026-09-06T16:28:45.474Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1444gitlab # [ 154.616646] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.475Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7516279efa790afe4731e1cd","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7516279efa790afe4731e1cd: start","job_status":"start","queue_duration_s":2.750399,"scheduling_latency_s":2.750413}1445gitlab # [ 154.617659] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.484Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7516279efa790afe4731e1cd","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7516279efa790afe4731e1cd: done: 0.009242 sec","job_status":"done","queue_duration_s":2.750399,"scheduling_latency_s":2.750413,"redis_calls":13,"redis_duration_s":0.001544,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005763,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009242,"completed_at":"2026-09-06T16:28:45.484Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1446gitlab # [ 154.653152] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4fcf4eb67e88ed1b69e7799c","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4fcf4eb67e88ed1b69e7799c: start","job_status":"start","queue_duration_s":2.760635,"scheduling_latency_s":2.760649}1447gitlab # [ 154.697941] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.497Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4fcf4eb67e88ed1b69e7799c","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4fcf4eb67e88ed1b69e7799c: done: 0.012409 sec","job_status":"done","queue_duration_s":2.760635,"scheduling_latency_s":2.760649,"redis_calls":13,"redis_duration_s":0.001936,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001415,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012409,"completed_at":"2026-09-06T16:28:45.497Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1448gitlab # [ 154.698931] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.501Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c6c5d30eb4c0b51cb2088c5","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c6c5d30eb4c0b51cb2088c5: start","job_status":"start","queue_duration_s":2.77647,"scheduling_latency_s":2.776485}1449gitlab # [ 154.710095] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.511Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c6c5d30eb4c0b51cb2088c5","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c6c5d30eb4c0b51cb2088c5: done: 0.009691 sec","job_status":"done","queue_duration_s":2.77647,"scheduling_latency_s":2.776485,"redis_calls":13,"redis_duration_s":0.001507,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000481,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001026,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006255,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009691,"completed_at":"2026-09-06T16:28:45.511Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1450gitlab # [ 154.764515] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.511Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92034dd422b3465a125feb2c","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92034dd422b3465a125feb2c: start","job_status":"start","queue_duration_s":2.787022,"scheduling_latency_s":2.787037}1451gitlab # [ 154.765463] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.521Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92034dd422b3465a125feb2c","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92034dd422b3465a125feb2c: done: 0.009207 sec","job_status":"done","queue_duration_s":2.787022,"scheduling_latency_s":2.787037,"redis_calls":13,"redis_duration_s":0.001535,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001026,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00583,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009207,"completed_at":"2026-09-06T16:28:45.521Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1452gitlab # [ 154.807087] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.522Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b213a535d25465e85980898e","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b213a535d25465e85980898e: start","job_status":"start","queue_duration_s":2.7972,"scheduling_latency_s":2.797213}1453gitlab # [ 154.807687] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.531Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b213a535d25465e85980898e","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b213a535d25465e85980898e: done: 0.009287 sec","job_status":"done","queue_duration_s":2.7972,"scheduling_latency_s":2.797213,"redis_calls":13,"redis_duration_s":0.001591,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001078,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009287,"completed_at":"2026-09-06T16:28:45.531Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1454gitlab # [ 154.851194] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.532Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0391afe26ca94f72ff715eb0","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0391afe26ca94f72ff715eb0: start","job_status":"start","queue_duration_s":2.807353,"scheduling_latency_s":2.807367}1455gitlab # [ 154.851940] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.541Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0391afe26ca94f72ff715eb0","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0391afe26ca94f72ff715eb0: done: 0.009409 sec","job_status":"done","queue_duration_s":2.807353,"scheduling_latency_s":2.807367,"redis_calls":13,"redis_duration_s":0.0016619999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001125,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009409,"completed_at":"2026-09-06T16:28:45.541Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1456gitlab # [ 154.882444] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.542Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4302554244efa710310754b9","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4302554244efa710310754b9: start","job_status":"start","queue_duration_s":2.817619,"scheduling_latency_s":2.817634}1457gitlab # [ 154.893839] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4302554244efa710310754b9","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4302554244efa710310754b9: done: 0.009364 sec","job_status":"done","queue_duration_s":2.817619,"scheduling_latency_s":2.817634,"redis_calls":13,"redis_duration_s":0.001673,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000604,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001069,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005988,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009364,"completed_at":"2026-09-06T16:28:45.551Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1458gitlab # [ 154.941199] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.552Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab8d77f13219ac240e4e549e","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab8d77f13219ac240e4e549e: start","job_status":"start","queue_duration_s":2.827824,"scheduling_latency_s":2.827839}1459gitlab # [ 154.941923] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab8d77f13219ac240e4e549e","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab8d77f13219ac240e4e549e: done: 0.009406 sec","job_status":"done","queue_duration_s":2.827824,"scheduling_latency_s":2.827839,"redis_calls":13,"redis_duration_s":0.001584,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001063,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005779,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009406,"completed_at":"2026-09-06T16:28:45.562Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1460gitlab # [ 154.991488] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.563Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c2d6c7272de486afee0490a","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c2d6c7272de486afee0490a: start","job_status":"start","queue_duration_s":2.838096,"scheduling_latency_s":2.838109}1461gitlab # [ 154.992178] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.572Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c2d6c7272de486afee0490a","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c2d6c7272de486afee0490a: done: 0.009315 sec","job_status":"done","queue_duration_s":2.838096,"scheduling_latency_s":2.838109,"redis_calls":13,"redis_duration_s":0.001617,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001077,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009315,"completed_at":"2026-09-06T16:28:45.572Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1462gitlab # [ 155.038452] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.573Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df5df04acecdc56aea23efd9","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df5df04acecdc56aea23efd9: start","job_status":"start","queue_duration_s":2.848259,"scheduling_latency_s":2.848272}1463gitlab # [ 155.039092] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df5df04acecdc56aea23efd9","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df5df04acecdc56aea23efd9: done: 0.00937 sec","job_status":"done","queue_duration_s":2.848259,"scheduling_latency_s":2.848272,"redis_calls":13,"redis_duration_s":0.001607,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00937,"completed_at":"2026-09-06T16:28:45.582Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1464gitlab # [ 155.088627] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"792d8cf91bb75a90e49c4d17","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-792d8cf91bb75a90e49c4d17: start","job_status":"start","queue_duration_s":2.858485,"scheduling_latency_s":2.858499}1465gitlab # [ 155.089546] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.592Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"792d8cf91bb75a90e49c4d17","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-792d8cf91bb75a90e49c4d17: done: 0.009386 sec","job_status":"done","queue_duration_s":2.858485,"scheduling_latency_s":2.858499,"redis_calls":13,"redis_duration_s":0.0015890000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001051,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009386,"completed_at":"2026-09-06T16:28:45.592Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1466gitlab # [ 155.112525] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"027224df7679d99691e0e5b5","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-027224df7679d99691e0e5b5: start","job_status":"start","queue_duration_s":2.868921,"scheduling_latency_s":2.868935}1467gitlab # [ 155.159653] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:28:45.603Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"027224df7679d99691e0e5b5","created_at":"2026-09-06T16:28:42.630Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-c53a9da9d0c84d10","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2eded348fcda76b6f03349ddb543b4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:28:42.724Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-027224df7679d99691e0e5b5: done: 0.009545 sec","job_status":"done","queue_duration_s":2.868921,"scheduling_latency_s":2.868935,"redis_calls":13,"redis_duration_s":0.001595,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005925,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009545,"completed_at":"2026-09-06T16:28:45.603Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1468gitlab # [ 163.174580] setup-gitlab-start[2234]: W, [2026-09-06T16:28:54.406080 #2234] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1469gitlab # [ 170.322128] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:01.553Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"f2ecba78c42ed3710eb817df","created_at":"2026-09-06T16:29:01.512Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea856654a499c0573331f95b11bbe113","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.551Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-f2ecba78c42ed3710eb817df: start","job_status":"start","queue_duration_s":0.002008,"scheduling_latency_s":0.002045}1470gitlab # [ 170.341299] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:01.573Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1471gitlab # [ 170.360415] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:01.592Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"f2ecba78c42ed3710eb817df","created_at":"2026-09-06T16:29:01.512Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea856654a499c0573331f95b11bbe113","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.551Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-f2ecba78c42ed3710eb817df: done: 0.038897 sec","job_status":"done","queue_duration_s":0.002008,"scheduling_latency_s":0.002045,"redis_calls":7,"redis_duration_s":0.009952,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.009336,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000616,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012133,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038897,"completed_at":"2026-09-06T16:29:01.592Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1472gitlab # [ 170.364466] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:01.596Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"3196d80fa2afc3c5a9343025","created_at":"2026-09-06T16:29:01.559Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"acc36651a80987695e78965667924e08","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:29:01.561Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-3196d80fa2afc3c5a9343025: start","job_status":"start","queue_duration_s":0.034791,"scheduling_latency_s":0.034826}1473gitlab # [ 170.406676] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:01.618Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1474gitlab # [ 170.407566] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:01.628Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"3196d80fa2afc3c5a9343025","created_at":"2026-09-06T16:29:01.559Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"acc36651a80987695e78965667924e08","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:29:01.561Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-3196d80fa2afc3c5a9343025: done: 0.031856 sec","job_status":"done","queue_duration_s":0.034791,"scheduling_latency_s":0.034826,"redis_calls":7,"redis_duration_s":0.00994,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008757,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001183,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010331,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031856,"completed_at":"2026-09-06T16:29:01.628Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1475gitlab # [ 170.408892] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:01.631Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"1af33b0838e748605e622549","created_at":"2026-09-06T16:29:01.571Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec884088985bfcec7bf43ed3307bf12c","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.574Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-1af33b0838e748605e622549: start","job_status":"start","queue_duration_s":0.056388,"scheduling_latency_s":0.056415}1476gitlab # [ 170.536398] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:01.768Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"1af33b0838e748605e622549","created_at":"2026-09-06T16:29:01.571Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec884088985bfcec7bf43ed3307bf12c","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.574Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-1af33b0838e748605e622549: done: 0.136814 sec","job_status":"done","queue_duration_s":0.056388,"scheduling_latency_s":0.056415,"redis_calls":7,"redis_duration_s":0.018399,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005417,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.012982,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.09,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.09,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017401,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.136814,"completed_at":"2026-09-06T16:29:01.767Z","load_balancing_strategy":"primary","db_duration_s":0.104487,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1477gitlab # [ 170.573983] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:01.771Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"1b6f4dad7c0fb8e4e6dd2f07","created_at":"2026-09-06T16:29:01.588Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"809094e945847141616a2730f3afbed9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.589Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-1b6f4dad7c0fb8e4e6dd2f07: start","job_status":"start","queue_duration_s":0.181894,"scheduling_latency_s":0.181936}1478gitlab # [ 170.574914] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:01.785Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1479gitlab # [ 170.575702] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:01.799Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"1b6f4dad7c0fb8e4e6dd2f07","created_at":"2026-09-06T16:29:01.588Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"809094e945847141616a2730f3afbed9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.589Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-1b6f4dad7c0fb8e4e6dd2f07: done: 0.027641 sec","job_status":"done","queue_duration_s":0.181894,"scheduling_latency_s":0.181936,"redis_calls":7,"redis_duration_s":0.002721,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001666,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001055,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011482,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027641,"completed_at":"2026-09-06T16:29:01.799Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1480gitlab # [ 170.632975] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:01.801Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"37b081309f0a7f5ca8dda799","created_at":"2026-09-06T16:29:01.597Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2e1688f0478fd220a297289d0888344b","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.598Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-37b081309f0a7f5ca8dda799: start","job_status":"start","queue_duration_s":0.203102,"scheduling_latency_s":0.203135}1481gitlab # [ 170.846440] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:02.078Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"37b081309f0a7f5ca8dda799","created_at":"2026-09-06T16:29:01.597Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2e1688f0478fd220a297289d0888344b","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.598Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-37b081309f0a7f5ca8dda799: done: 0.276853 sec","job_status":"done","queue_duration_s":0.203102,"scheduling_latency_s":0.203135,"redis_calls":562,"redis_duration_s":0.106325,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001043,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.105282,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.192107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.276853,"completed_at":"2026-09-06T16:29:02.078Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1482gitlab # [ 170.924172] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:02.156Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"06619b825a0d8a511de992fe","created_at":"2026-09-06T16:29:01.601Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"02bddb1045e13e33f46595c7ea921cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.602Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-06619b825a0d8a511de992fe: start","job_status":"start","queue_duration_s":0.553036,"scheduling_latency_s":0.553086}1483gitlab # [ 170.955202] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:02.186Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"06619b825a0d8a511de992fe","created_at":"2026-09-06T16:29:01.601Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"02bddb1045e13e33f46595c7ea921cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.602Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-06619b825a0d8a511de992fe: done: 0.031012 sec","job_status":"done","queue_duration_s":0.553036,"scheduling_latency_s":0.553086,"redis_calls":7,"redis_duration_s":0.0015300000000000001,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001126,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000404,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012899,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031012,"completed_at":"2026-09-06T16:29:02.186Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1484gitlab # [ 170.993738] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:02.225Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"3ad3a9cf60677dcfe9325dad","created_at":"2026-09-06T16:29:01.617Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0fdceba9156aa29c8b1868d5d74e2b3d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.622Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-3ad3a9cf60677dcfe9325dad: start","job_status":"start","queue_duration_s":0.603211,"scheduling_latency_s":0.603268}1485gitlab # [ 171.214274] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:02.446Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"3ad3a9cf60677dcfe9325dad","created_at":"2026-09-06T16:29:01.617Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0fdceba9156aa29c8b1868d5d74e2b3d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.622Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-3ad3a9cf60677dcfe9325dad: done: 0.220762 sec","job_status":"done","queue_duration_s":0.603211,"scheduling_latency_s":0.603268,"redis_calls":9,"redis_duration_s":0.002411,"redis_read_bytes":7,"redis_write_bytes":987,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001214,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001197,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":129,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":129,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.041,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.041,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.140776,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000810997,"exclusive_lock_hold_duration_s":0.188554132,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.220762,"completed_at":"2026-09-06T16:29:02.446Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.076418,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1486gitlab # [ 171.292277] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:02.524Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"bf5f3d98fda2f26796138c25","created_at":"2026-09-06T16:29:01.628Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"806bde1b519e9683d460470e0e9ffd39","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:29:01.630Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-bf5f3d98fda2f26796138c25: start","job_status":"start","queue_duration_s":0.89376,"scheduling_latency_s":0.893797}1487gitlab # [ 171.313414] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:02.545Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"bf5f3d98fda2f26796138c25","created_at":"2026-09-06T16:29:01.628Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"806bde1b519e9683d460470e0e9ffd39","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:29:01.630Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-bf5f3d98fda2f26796138c25: done: 0.021138 sec","job_status":"done","queue_duration_s":0.89376,"scheduling_latency_s":0.893797,"redis_calls":7,"redis_duration_s":0.001789,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001286,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000503,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012863,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021138,"completed_at":"2026-09-06T16:29:02.545Z","load_balancing_strategy":"primary","db_duration_s":0.002195,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1488gitlab # [ 171.350786] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:02.581Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"2eeb246d2fe0bf411ddc333f","created_at":"2026-09-06T16:29:01.635Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2333a3b55b045ce9e482cece3044c2e8","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.636Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-2eeb246d2fe0bf411ddc333f: start","job_status":"start","queue_duration_s":0.945149,"scheduling_latency_s":0.945187}1489gitlab # [ 171.370895] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:02.602Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"2eeb246d2fe0bf411ddc333f","created_at":"2026-09-06T16:29:01.635Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2333a3b55b045ce9e482cece3044c2e8","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.636Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-2eeb246d2fe0bf411ddc333f: done: 0.020607 sec","job_status":"done","queue_duration_s":0.945149,"scheduling_latency_s":0.945187,"redis_calls":7,"redis_duration_s":0.002193,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0016,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000593,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020607,"completed_at":"2026-09-06T16:29:02.602Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001659,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1490gitlab # [ 171.409362] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:02.641Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"a2d80ec06728aa959eadc8cb","created_at":"2026-09-06T16:29:01.652Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9461f5a90391efa016ba7ed4254f58eb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.658Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-a2d80ec06728aa959eadc8cb: start","job_status":"start","queue_duration_s":0.983232,"scheduling_latency_s":0.983272}1491gitlab # [ 171.444278] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:02.675Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"a2d80ec06728aa959eadc8cb","created_at":"2026-09-06T16:29:01.652Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9461f5a90391efa016ba7ed4254f58eb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.658Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-a2d80ec06728aa959eadc8cb: done: 0.034704 sec","job_status":"done","queue_duration_s":0.983232,"scheduling_latency_s":0.983272,"redis_calls":7,"redis_duration_s":0.0033940000000000003,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001347,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002047,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034704,"completed_at":"2026-09-06T16:29:02.675Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1492gitlab # [ 171.482261] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:02.714Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"81ec2b80f644d8bbf8a17d86","created_at":"2026-09-06T16:29:01.671Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"184e5c939af2ae876ac003297bacd41e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.673Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-81ec2b80f644d8bbf8a17d86: start","job_status":"start","queue_duration_s":1.040587,"scheduling_latency_s":1.040628}1493gitlab # [ 171.493593] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:02.725Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1494gitlab # [ 171.501721] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:02.733Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"81ec2b80f644d8bbf8a17d86","created_at":"2026-09-06T16:29:01.671Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"184e5c939af2ae876ac003297bacd41e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.673Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-81ec2b80f644d8bbf8a17d86: done: 0.019501 sec","job_status":"done","queue_duration_s":1.040587,"scheduling_latency_s":1.040628,"redis_calls":7,"redis_duration_s":0.002293,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001659,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000634,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011477,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019501,"completed_at":"2026-09-06T16:29:02.733Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1495gitlab # [ 171.540084] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:02.772Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"e31c73b4dd55a3dde41b2cd3","created_at":"2026-09-06T16:29:01.686Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2490fb639e56afce16be103927c8b59f","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.689Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-e31c73b4dd55a3dde41b2cd3: start","job_status":"start","queue_duration_s":1.082914,"scheduling_latency_s":1.082956}1496gitlab # [ 171.578073] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:02.809Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"e31c73b4dd55a3dde41b2cd3","created_at":"2026-09-06T16:29:01.686Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2490fb639e56afce16be103927c8b59f","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.689Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-e31c73b4dd55a3dde41b2cd3: done: 0.037938 sec","job_status":"done","queue_duration_s":1.082914,"scheduling_latency_s":1.082956,"redis_calls":7,"redis_duration_s":0.001424,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001049,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000375,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015979,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037938,"completed_at":"2026-09-06T16:29:02.809Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004819,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1497gitlab # [ 171.616100] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:02.848Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"2e241c262cde4274eeff626f","created_at":"2026-09-06T16:29:01.692Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d912057a4eae49a3e8f7e1a1ae74a83","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.694Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-2e241c262cde4274eeff626f: start","job_status":"start","queue_duration_s":1.153905,"scheduling_latency_s":1.153949}1498gitlab # [ 171.673359] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:02.904Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"2e241c262cde4274eeff626f","created_at":"2026-09-06T16:29:01.692Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d912057a4eae49a3e8f7e1a1ae74a83","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.694Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-2e241c262cde4274eeff626f: done: 0.056427 sec","job_status":"done","queue_duration_s":1.153905,"scheduling_latency_s":1.153949,"redis_calls":9,"redis_duration_s":0.0024460000000000003,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001417,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001029,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026961,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000736966,"exclusive_lock_hold_duration_s":0.02427599,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.056427,"completed_at":"2026-09-06T16:29:02.904Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011459,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1499gitlab # [ 171.677439] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:02.909Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"76ea6ab07cf345bcf2ea67e2","created_at":"2026-09-06T16:29:01.753Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"73bbe480ea7961124a5c0347c6df3d03","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.757Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-76ea6ab07cf345bcf2ea67e2: start","job_status":"start","queue_duration_s":1.151885,"scheduling_latency_s":1.151921}1500gitlab # [ 171.710676] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:02.942Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"76ea6ab07cf345bcf2ea67e2","created_at":"2026-09-06T16:29:01.753Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"73bbe480ea7961124a5c0347c6df3d03","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.757Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-76ea6ab07cf345bcf2ea67e2: done: 0.033454 sec","job_status":"done","queue_duration_s":1.151885,"scheduling_latency_s":1.151921,"redis_calls":14,"redis_duration_s":0.002692,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000262,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001667,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000763,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013291,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033454,"completed_at":"2026-09-06T16:29:02.942Z","load_balancing_strategy":"primary","db_duration_s":0.009249,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1501gitlab # [ 171.721123] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:02.943Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"5a91d83c8a351a3143ab6ea5","created_at":"2026-09-06T16:29:01.766Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c1b3e5bad4b5c7f45b2da564eb5e92bb","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.772Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-5a91d83c8a351a3143ab6ea5: start","job_status":"start","queue_duration_s":1.171159,"scheduling_latency_s":1.171176}1502gitlab # [ 171.764936] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:02.956Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"5a91d83c8a351a3143ab6ea5","created_at":"2026-09-06T16:29:01.766Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c1b3e5bad4b5c7f45b2da564eb5e92bb","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.772Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-5a91d83c8a351a3143ab6ea5: done: 0.012502 sec","job_status":"done","queue_duration_s":1.171159,"scheduling_latency_s":1.171176,"redis_calls":9,"redis_duration_s":0.001238,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00065,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000588,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000328254,"exclusive_lock_hold_duration_s":0.004108902,"duration_s":0.012502,"completed_at":"2026-09-06T16:29:02.956Z","load_balancing_strategy":"primary","db_duration_s":0.002544,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1503gitlab # [ 171.768598] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:02.957Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"73990bb25b5c42ca785df079","created_at":"2026-09-06T16:29:01.792Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.797Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-73990bb25b5c42ca785df079: start","job_status":"start","queue_duration_s":1.159968,"scheduling_latency_s":1.159984}1504gitlab # [ 171.781331] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.009Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"73990bb25b5c42ca785df079","created_at":"2026-09-06T16:29:01.792Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.797Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-73990bb25b5c42ca785df079: done: 0.051962 sec","job_status":"done","queue_duration_s":1.159968,"scheduling_latency_s":1.159984,"redis_calls":60,"redis_duration_s":0.008293,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.000575,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.00733,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000388,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043502,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.051962,"completed_at":"2026-09-06T16:29:03.009Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1505gitlab # [ 171.826997] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.010Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"28069ec46184c45d0d784bc5","created_at":"2026-09-06T16:29:01.804Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6f423174a4e0d1e6f8025738982b9c19","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.810Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-28069ec46184c45d0d784bc5: start","job_status":"start","queue_duration_s":1.199589,"scheduling_latency_s":1.199608}1506gitlab # [ 171.829867] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.025Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"28069ec46184c45d0d784bc5","created_at":"2026-09-06T16:29:01.804Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6f423174a4e0d1e6f8025738982b9c19","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.810Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-28069ec46184c45d0d784bc5: done: 0.015176 sec","job_status":"done","queue_duration_s":1.199589,"scheduling_latency_s":1.199608,"redis_calls":10,"redis_duration_s":0.001408,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000824,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000584,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.005221334,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.015176,"completed_at":"2026-09-06T16:29:03.025Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003573,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1507gitlab # [ 171.877318] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.026Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"d3b11350d07cf46dc4c01b6b","created_at":"2026-09-06T16:29:01.816Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed36a11ddfc1a9d639b4bf373050ebee","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-06T16:29:01.829Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-d3b11350d07cf46dc4c01b6b: start","job_status":"start","queue_duration_s":1.19662,"scheduling_latency_s":1.196634}1508gitlab # [ 171.879291] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.030Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1509gitlab # [ 171.882511] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.033Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"d3b11350d07cf46dc4c01b6b","created_at":"2026-09-06T16:29:01.816Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed36a11ddfc1a9d639b4bf373050ebee","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-06T16:29:01.829Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-d3b11350d07cf46dc4c01b6b: done: 0.007535 sec","job_status":"done","queue_duration_s":1.19662,"scheduling_latency_s":1.196634,"redis_calls":7,"redis_duration_s":0.0009059999999999999,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000659,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000247,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007535,"completed_at":"2026-09-06T16:29:03.033Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1510gitlab # [ 171.934272] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.034Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"58375519701761baa8561c4d","created_at":"2026-09-06T16:29:01.836Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"60703a3b29b544c0961c68d911edc2fd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.838Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-58375519701761baa8561c4d: start","job_status":"start","queue_duration_s":1.195702,"scheduling_latency_s":1.195716}1511gitlab # [ 171.935057] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.059Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"58375519701761baa8561c4d","created_at":"2026-09-06T16:29:01.836Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"60703a3b29b544c0961c68d911edc2fd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.838Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-58375519701761baa8561c4d: done: 0.025383 sec","job_status":"done","queue_duration_s":1.195702,"scheduling_latency_s":1.195716,"redis_calls":9,"redis_duration_s":0.001353,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000723,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00063,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010591,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00034753,"exclusive_lock_hold_duration_s":0.011500903,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.025383,"completed_at":"2026-09-06T16:29:03.059Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011304,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1512gitlab # [ 172.007554] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.060Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"f3a69c654363d5e57624fa7f","created_at":"2026-09-06T16:29:01.849Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8b86951097316db7ddb7a3cab2349c3","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.851Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-f3a69c654363d5e57624fa7f: start","job_status":"start","queue_duration_s":1.209104,"scheduling_latency_s":1.209118}1513gitlab # [ 172.008488] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.072Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"f3a69c654363d5e57624fa7f","created_at":"2026-09-06T16:29:01.849Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8b86951097316db7ddb7a3cab2349c3","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.851Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-f3a69c654363d5e57624fa7f: done: 0.012039 sec","job_status":"done","queue_duration_s":1.209104,"scheduling_latency_s":1.209118,"redis_calls":9,"redis_duration_s":0.0012100000000000001,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000637,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000573,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005929,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000305067,"exclusive_lock_hold_duration_s":0.003749359,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.012039,"completed_at":"2026-09-06T16:29:03.072Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002667,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1514gitlab # [ 172.046799] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.073Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"4bbb6112e17527dfb88d90ac","created_at":"2026-09-06T16:29:01.858Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c43299b960ff500941267a41d879196a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.860Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-4bbb6112e17527dfb88d90ac: start","job_status":"start","queue_duration_s":1.213858,"scheduling_latency_s":1.213872}1515gitlab # [ 172.047514] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.078Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1516gitlab # [ 172.048067] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.081Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"4bbb6112e17527dfb88d90ac","created_at":"2026-09-06T16:29:01.858Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-2c9820eb108a4111","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c43299b960ff500941267a41d879196a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:01.860Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-4bbb6112e17527dfb88d90ac: done: 0.0076 sec","job_status":"done","queue_duration_s":1.213858,"scheduling_latency_s":1.213872,"redis_calls":7,"redis_duration_s":0.001031,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000638,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000393,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0076,"completed_at":"2026-09-06T16:29:03.081Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1517gitlab # [ 172.097664] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.082Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b8bab7ad254a6a587e2da126","created_at":"2026-09-06T16:29:02.932Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-0a8d2cb067c64e99","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"73bbe480ea7961124a5c0347c6df3d03","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:02.938Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b8bab7ad254a6a587e2da126: start","job_status":"start","queue_duration_s":0.143373,"scheduling_latency_s":0.143387}1518gitlab # [ 172.098386] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.089Z","message":"No existing merge request to be cleaned up."}1519gitlab # [ 172.098885] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.094Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b8bab7ad254a6a587e2da126","created_at":"2026-09-06T16:29:02.932Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-0a8d2cb067c64e99","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"73bbe480ea7961124a5c0347c6df3d03","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:02.938Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b8bab7ad254a6a587e2da126: done: 0.011822 sec","job_status":"done","queue_duration_s":0.143373,"scheduling_latency_s":0.143387,"redis_calls":10,"redis_duration_s":0.001317,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000791,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007581,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011822,"completed_at":"2026-09-06T16:29:03.094Z","load_balancing_strategy":"primary","db_duration_s":0.001556,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1520gitlab # [ 172.114715] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.095Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"47a6cc7ef843e4463bf2fc35","created_at":"2026-09-06T16:29:02.932Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-0a8d2cb067c64e99","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"73bbe480ea7961124a5c0347c6df3d03","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:02.938Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-47a6cc7ef843e4463bf2fc35: start","job_status":"start","queue_duration_s":0.156135,"scheduling_latency_s":0.156149}1521gitlab # [ 172.125658] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.101Z","message":"No existing merge request to be cleaned up."}1522gitlab # [ 172.126888] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.106Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"47a6cc7ef843e4463bf2fc35","created_at":"2026-09-06T16:29:02.932Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-0a8d2cb067c64e99","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"73bbe480ea7961124a5c0347c6df3d03","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:02.938Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-47a6cc7ef843e4463bf2fc35: done: 0.010962 sec","job_status":"done","queue_duration_s":0.156135,"scheduling_latency_s":0.156149,"redis_calls":10,"redis_duration_s":0.001343,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000799,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010962,"completed_at":"2026-09-06T16:29:03.106Z","load_balancing_strategy":"primary","db_duration_s":0.001372,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1523gitlab # [ 172.184634] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.106Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bc0b62339d09131f7306e528","created_at":"2026-09-06T16:29:02.932Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-0a8d2cb067c64e99","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"73bbe480ea7961124a5c0347c6df3d03","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:02.938Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bc0b62339d09131f7306e528: start","job_status":"start","queue_duration_s":0.167947,"scheduling_latency_s":0.167961}1524gitlab # [ 172.185227] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.115Z","message":"No existing merge request to be cleaned up."}1525gitlab # [ 172.185815] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.121Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bc0b62339d09131f7306e528","created_at":"2026-09-06T16:29:02.932Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-0a8d2cb067c64e99","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"73bbe480ea7961124a5c0347c6df3d03","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:02.938Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bc0b62339d09131f7306e528: done: 0.014506 sec","job_status":"done","queue_duration_s":0.167947,"scheduling_latency_s":0.167961,"redis_calls":10,"redis_duration_s":0.001376,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00083,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00724,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014506,"completed_at":"2026-09-06T16:29:03.121Z","load_balancing_strategy":"primary","db_duration_s":0.005676,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1526gitlab # [ 172.205054] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.122Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"65892cab4c5795cd0d13fff3","created_at":"2026-09-06T16:29:02.932Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-0a8d2cb067c64e99","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"73bbe480ea7961124a5c0347c6df3d03","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:02.938Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-65892cab4c5795cd0d13fff3: start","job_status":"start","queue_duration_s":0.183319,"scheduling_latency_s":0.183334}1527gitlab # [ 172.242234] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.129Z","message":"No existing merge request to be cleaned up."}1528gitlab # [ 172.242820] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.134Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"65892cab4c5795cd0d13fff3","created_at":"2026-09-06T16:29:02.932Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-0a8d2cb067c64e99","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"73bbe480ea7961124a5c0347c6df3d03","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:02.938Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-65892cab4c5795cd0d13fff3: done: 0.011865 sec","job_status":"done","queue_duration_s":0.183319,"scheduling_latency_s":0.183334,"redis_calls":10,"redis_duration_s":0.001283,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000769,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011865,"completed_at":"2026-09-06T16:29:03.134Z","load_balancing_strategy":"primary","db_duration_s":0.002902,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1529gitlab # [ 172.249583] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.135Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66bd590c660532e8c8241e4e","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66bd590c660532e8c8241e4e: start","job_status":"start","queue_duration_s":0.130759,"scheduling_latency_s":0.130774}1530gitlab # [ 172.287650] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.144Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66bd590c660532e8c8241e4e","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66bd590c660532e8c8241e4e: done: 0.009441 sec","job_status":"done","queue_duration_s":0.130759,"scheduling_latency_s":0.130774,"redis_calls":13,"redis_duration_s":0.001579,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005901,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009441,"completed_at":"2026-09-06T16:29:03.144Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1531gitlab # [ 172.293433] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.145Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1638022c82f34519e53c2e83","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1638022c82f34519e53c2e83: start","job_status":"start","queue_duration_s":0.141148,"scheduling_latency_s":0.141163}1532gitlab # [ 172.304098] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.154Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1638022c82f34519e53c2e83","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1638022c82f34519e53c2e83: done: 0.009216 sec","job_status":"done","queue_duration_s":0.141148,"scheduling_latency_s":0.141163,"redis_calls":13,"redis_duration_s":0.001582,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005861,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009216,"completed_at":"2026-09-06T16:29:03.154Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1533gitlab # [ 172.342150] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.155Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a0c8bbd91b06bc814cf639d","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a0c8bbd91b06bc814cf639d: start","job_status":"start","queue_duration_s":0.151182,"scheduling_latency_s":0.151196}1534gitlab # [ 172.348430] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.164Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a0c8bbd91b06bc814cf639d","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a0c8bbd91b06bc814cf639d: done: 0.009193 sec","job_status":"done","queue_duration_s":0.151182,"scheduling_latency_s":0.151196,"redis_calls":13,"redis_duration_s":0.001604,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001083,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009193,"completed_at":"2026-09-06T16:29:03.164Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1535gitlab # [ 172.402946] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f50c0c5c993f017ce5078f1","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f50c0c5c993f017ce5078f1: start","job_status":"start","queue_duration_s":0.161197,"scheduling_latency_s":0.161211}1536gitlab # [ 172.403683] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.174Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f50c0c5c993f017ce5078f1","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f50c0c5c993f017ce5078f1: done: 0.009272 sec","job_status":"done","queue_duration_s":0.161197,"scheduling_latency_s":0.161211,"redis_calls":13,"redis_duration_s":0.001695,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001193,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009272,"completed_at":"2026-09-06T16:29:03.174Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1537gitlab # [ 172.452171] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.175Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81452f7832485cead6d19f8e","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81452f7832485cead6d19f8e: start","job_status":"start","queue_duration_s":0.171444,"scheduling_latency_s":0.171458}1538gitlab # [ 172.452849] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.184Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81452f7832485cead6d19f8e","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81452f7832485cead6d19f8e: done: 0.009147 sec","job_status":"done","queue_duration_s":0.171444,"scheduling_latency_s":0.171458,"redis_calls":13,"redis_duration_s":0.001532,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005857,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009147,"completed_at":"2026-09-06T16:29:03.184Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1539gitlab # [ 172.501823] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.185Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d4525a3ae7f8949c13848c3","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d4525a3ae7f8949c13848c3: start","job_status":"start","queue_duration_s":0.181426,"scheduling_latency_s":0.181438}1540gitlab # [ 172.502571] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.194Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d4525a3ae7f8949c13848c3","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d4525a3ae7f8949c13848c3: done: 0.009164 sec","job_status":"done","queue_duration_s":0.181426,"scheduling_latency_s":0.181438,"redis_calls":13,"redis_duration_s":0.001742,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001196,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00587,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009164,"completed_at":"2026-09-06T16:29:03.194Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1541gitlab # [ 172.512816] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.195Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62e1577e5dd00a0ab7bb9005","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62e1577e5dd00a0ab7bb9005: start","job_status":"start","queue_duration_s":0.191427,"scheduling_latency_s":0.19144}1542gitlab # [ 172.551114] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.204Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62e1577e5dd00a0ab7bb9005","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62e1577e5dd00a0ab7bb9005: done: 0.009179 sec","job_status":"done","queue_duration_s":0.191427,"scheduling_latency_s":0.19144,"redis_calls":13,"redis_duration_s":0.001605,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005868,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009179,"completed_at":"2026-09-06T16:29:03.204Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1543gitlab # [ 172.556197] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd899d68afec6a8a836d4bb1","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd899d68afec6a8a836d4bb1: start","job_status":"start","queue_duration_s":0.20145,"scheduling_latency_s":0.201473}1544gitlab # [ 172.599789] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.214Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd899d68afec6a8a836d4bb1","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd899d68afec6a8a836d4bb1: done: 0.009195 sec","job_status":"done","queue_duration_s":0.20145,"scheduling_latency_s":0.201473,"redis_calls":13,"redis_duration_s":0.001657,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001149,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005754,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009195,"completed_at":"2026-09-06T16:29:03.214Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1545gitlab # [ 172.600833] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.215Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f88dc395c92bdb0cfad70fb4","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f88dc395c92bdb0cfad70fb4: start","job_status":"start","queue_duration_s":0.211673,"scheduling_latency_s":0.211686}1546gitlab # [ 172.610734] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.225Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f88dc395c92bdb0cfad70fb4","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f88dc395c92bdb0cfad70fb4: done: 0.009137 sec","job_status":"done","queue_duration_s":0.211673,"scheduling_latency_s":0.211686,"redis_calls":13,"redis_duration_s":0.001581,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001064,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005874,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009137,"completed_at":"2026-09-06T16:29:03.225Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1547gitlab # [ 172.656704] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.225Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"611044361c2f4f7b52544f09","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-611044361c2f4f7b52544f09: start","job_status":"start","queue_duration_s":0.221645,"scheduling_latency_s":0.221659}1548gitlab # [ 172.657488] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.235Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"611044361c2f4f7b52544f09","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-611044361c2f4f7b52544f09: done: 0.009221 sec","job_status":"done","queue_duration_s":0.221645,"scheduling_latency_s":0.221659,"redis_calls":13,"redis_duration_s":0.0017590000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000705,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009221,"completed_at":"2026-09-06T16:29:03.235Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1549gitlab # [ 172.701676] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.236Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afe884b1b1c8989774cbc9d8","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afe884b1b1c8989774cbc9d8: start","job_status":"start","queue_duration_s":0.231733,"scheduling_latency_s":0.231746}1550gitlab # [ 172.702453] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.245Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afe884b1b1c8989774cbc9d8","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afe884b1b1c8989774cbc9d8: done: 0.009135 sec","job_status":"done","queue_duration_s":0.231733,"scheduling_latency_s":0.231746,"redis_calls":13,"redis_duration_s":0.001627,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000569,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001058,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009135,"completed_at":"2026-09-06T16:29:03.245Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1551gitlab # [ 172.748054] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.246Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"807b3b632ac13e806318101f","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-807b3b632ac13e806318101f: start","job_status":"start","queue_duration_s":0.241895,"scheduling_latency_s":0.24191}1552gitlab # [ 172.748694] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.255Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"807b3b632ac13e806318101f","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-807b3b632ac13e806318101f: done: 0.009137 sec","job_status":"done","queue_duration_s":0.241895,"scheduling_latency_s":0.24191,"redis_calls":13,"redis_duration_s":0.001542,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005877,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009137,"completed_at":"2026-09-06T16:29:03.255Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1553gitlab # [ 172.790744] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.256Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc60a546008d098142d761f7","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc60a546008d098142d761f7: start","job_status":"start","queue_duration_s":0.251874,"scheduling_latency_s":0.251887}1554gitlab # [ 172.791842] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.265Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc60a546008d098142d761f7","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc60a546008d098142d761f7: done: 0.009217 sec","job_status":"done","queue_duration_s":0.251874,"scheduling_latency_s":0.251887,"redis_calls":13,"redis_duration_s":0.0015580000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001048,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009217,"completed_at":"2026-09-06T16:29:03.265Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1555gitlab # [ 172.828918] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.266Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"521da8a2fe87b9de5876cc73","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-521da8a2fe87b9de5876cc73: start","job_status":"start","queue_duration_s":0.262095,"scheduling_latency_s":0.262108}1556gitlab # [ 172.840672] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.275Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"521da8a2fe87b9de5876cc73","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-521da8a2fe87b9de5876cc73: done: 0.009358 sec","job_status":"done","queue_duration_s":0.262095,"scheduling_latency_s":0.262108,"redis_calls":13,"redis_duration_s":0.0018839999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000669,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001215,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005871,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009358,"completed_at":"2026-09-06T16:29:03.275Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1557gitlab # [ 172.902550] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.276Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9110c9035c0a4b6c5b458574","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9110c9035c0a4b6c5b458574: start","job_status":"start","queue_duration_s":0.272309,"scheduling_latency_s":0.272323}1558gitlab # [ 172.903234] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9110c9035c0a4b6c5b458574","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9110c9035c0a4b6c5b458574: done: 0.009264 sec","job_status":"done","queue_duration_s":0.272309,"scheduling_latency_s":0.272323,"redis_calls":13,"redis_duration_s":0.001588,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001042,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009264,"completed_at":"2026-09-06T16:29:03.285Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1559gitlab # [ 172.948544] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.286Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14118258fea1bcd1fa9bd308","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14118258fea1bcd1fa9bd308: start","job_status":"start","queue_duration_s":0.282598,"scheduling_latency_s":0.282612}1560gitlab # [ 172.949461] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.296Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14118258fea1bcd1fa9bd308","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14118258fea1bcd1fa9bd308: done: 0.009278 sec","job_status":"done","queue_duration_s":0.282598,"scheduling_latency_s":0.282612,"redis_calls":13,"redis_duration_s":0.0016189999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009278,"completed_at":"2026-09-06T16:29:03.296Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1561gitlab # [ 172.993740] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.297Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f6ce67abe40eea1f8e1cb8d","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f6ce67abe40eea1f8e1cb8d: start","job_status":"start","queue_duration_s":0.292735,"scheduling_latency_s":0.292749}1562gitlab # [ 172.994538] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.306Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f6ce67abe40eea1f8e1cb8d","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f6ce67abe40eea1f8e1cb8d: done: 0.009083 sec","job_status":"done","queue_duration_s":0.292735,"scheduling_latency_s":0.292749,"redis_calls":13,"redis_duration_s":0.0015639999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00104,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005824,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009083,"completed_at":"2026-09-06T16:29:03.306Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1563gitlab # [ 173.015524] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.306Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee70e6e3bad788d0f452380e","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee70e6e3bad788d0f452380e: start","job_status":"start","queue_duration_s":0.302655,"scheduling_latency_s":0.302668}1564gitlab # [ 173.026712] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.316Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee70e6e3bad788d0f452380e","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee70e6e3bad788d0f452380e: done: 0.009246 sec","job_status":"done","queue_duration_s":0.302655,"scheduling_latency_s":0.302668,"redis_calls":13,"redis_duration_s":0.0015890000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001056,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005981,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009246,"completed_at":"2026-09-06T16:29:03.316Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1565gitlab # [ 173.066524] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.317Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c56ad1760e3e9ff08b620830","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c56ad1760e3e9ff08b620830: start","job_status":"start","queue_duration_s":0.312926,"scheduling_latency_s":0.312939}1566gitlab # [ 173.073038] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.326Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c56ad1760e3e9ff08b620830","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c56ad1760e3e9ff08b620830: done: 0.00928 sec","job_status":"done","queue_duration_s":0.312926,"scheduling_latency_s":0.312939,"redis_calls":13,"redis_duration_s":0.001837,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000706,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001131,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005905,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00928,"completed_at":"2026-09-06T16:29:03.326Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1567gitlab # [ 173.139738] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.327Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77d6840fba3bceb2d28a8032","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77d6840fba3bceb2d28a8032: start","job_status":"start","queue_duration_s":0.323041,"scheduling_latency_s":0.323055}1568gitlab # [ 173.140621] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.336Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77d6840fba3bceb2d28a8032","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77d6840fba3bceb2d28a8032: done: 0.009138 sec","job_status":"done","queue_duration_s":0.323041,"scheduling_latency_s":0.323055,"redis_calls":13,"redis_duration_s":0.001518,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001015,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009138,"completed_at":"2026-09-06T16:29:03.336Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1569gitlab # [ 173.190607] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.337Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dabf36b929c84b9b262c8f1f","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dabf36b929c84b9b262c8f1f: start","job_status":"start","queue_duration_s":0.33303,"scheduling_latency_s":0.333045}1570gitlab # [ 173.191163] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.346Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dabf36b929c84b9b262c8f1f","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dabf36b929c84b9b262c8f1f: done: 0.009159 sec","job_status":"done","queue_duration_s":0.33303,"scheduling_latency_s":0.333045,"redis_calls":13,"redis_duration_s":0.0015480000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001051,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005878,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009159,"completed_at":"2026-09-06T16:29:03.346Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1571gitlab # [ 173.232643] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.347Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"109238af687b82967c706973","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-109238af687b82967c706973: start","job_status":"start","queue_duration_s":0.343035,"scheduling_latency_s":0.343048}1572gitlab # [ 173.233296] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.356Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"109238af687b82967c706973","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-109238af687b82967c706973: done: 0.009408 sec","job_status":"done","queue_duration_s":0.343035,"scheduling_latency_s":0.343048,"redis_calls":13,"redis_duration_s":0.001506,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001011,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005752,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009408,"completed_at":"2026-09-06T16:29:03.356Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1573gitlab # [ 173.276647] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.357Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"590d7edd0ae2ade28ed1cf8b","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-590d7edd0ae2ade28ed1cf8b: start","job_status":"start","queue_duration_s":0.353284,"scheduling_latency_s":0.353297}1574gitlab # [ 173.277590] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.366Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"590d7edd0ae2ade28ed1cf8b","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-590d7edd0ae2ade28ed1cf8b: done: 0.009267 sec","job_status":"done","queue_duration_s":0.353284,"scheduling_latency_s":0.353297,"redis_calls":13,"redis_duration_s":0.0016769999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001174,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009267,"completed_at":"2026-09-06T16:29:03.366Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1575gitlab # [ 173.293610] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.367Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea571ac32c4c1a8f0ad43324","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea571ac32c4c1a8f0ad43324: start","job_status":"start","queue_duration_s":0.363552,"scheduling_latency_s":0.363566}1576gitlab # [ 173.305028] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.377Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea571ac32c4c1a8f0ad43324","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea571ac32c4c1a8f0ad43324: done: 0.009691 sec","job_status":"done","queue_duration_s":0.363552,"scheduling_latency_s":0.363566,"redis_calls":13,"redis_duration_s":0.0018989999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001365,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00619,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009691,"completed_at":"2026-09-06T16:29:03.377Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1577gitlab # [ 173.357960] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06d13047bcaa5426b48e4081","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06d13047bcaa5426b48e4081: start","job_status":"start","queue_duration_s":0.374094,"scheduling_latency_s":0.374108}1578gitlab # [ 173.358655] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.391Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06d13047bcaa5426b48e4081","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06d13047bcaa5426b48e4081: done: 0.013489 sec","job_status":"done","queue_duration_s":0.374094,"scheduling_latency_s":0.374108,"redis_calls":13,"redis_duration_s":0.0015300000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001024,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013489,"completed_at":"2026-09-06T16:29:03.391Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1579gitlab # [ 173.411525] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.392Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"249e7c8e38846014868b2092","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-249e7c8e38846014868b2092: start","job_status":"start","queue_duration_s":0.388432,"scheduling_latency_s":0.388446}1580gitlab # [ 173.412296] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.401Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"249e7c8e38846014868b2092","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-249e7c8e38846014868b2092: done: 0.009117 sec","job_status":"done","queue_duration_s":0.388432,"scheduling_latency_s":0.388446,"redis_calls":13,"redis_duration_s":0.001562,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005811,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009117,"completed_at":"2026-09-06T16:29:03.401Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1581gitlab # [ 173.469508] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.402Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bffa0c0a306173f5036996bd","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bffa0c0a306173f5036996bd: start","job_status":"start","queue_duration_s":0.398387,"scheduling_latency_s":0.398403}1582gitlab # [ 173.470307] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.411Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bffa0c0a306173f5036996bd","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bffa0c0a306173f5036996bd: done: 0.009021 sec","job_status":"done","queue_duration_s":0.398387,"scheduling_latency_s":0.398403,"redis_calls":13,"redis_duration_s":0.001617,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001043,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005742,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009021,"completed_at":"2026-09-06T16:29:03.411Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1583gitlab # [ 173.510488] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.412Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0da99d4c5639db8f815a7faf","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0da99d4c5639db8f815a7faf: start","job_status":"start","queue_duration_s":0.408238,"scheduling_latency_s":0.408275}1584gitlab # [ 173.511272] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.421Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0da99d4c5639db8f815a7faf","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0da99d4c5639db8f815a7faf: done: 0.00898 sec","job_status":"done","queue_duration_s":0.408238,"scheduling_latency_s":0.408275,"redis_calls":13,"redis_duration_s":0.001598,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001057,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00898,"completed_at":"2026-09-06T16:29:03.421Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1585gitlab # [ 173.553866] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.422Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ccf5155761f3b2c609a3a625","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ccf5155761f3b2c609a3a625: start","job_status":"start","queue_duration_s":0.418056,"scheduling_latency_s":0.41807}1586gitlab # [ 173.554647] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ccf5155761f3b2c609a3a625","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ccf5155761f3b2c609a3a625: done: 0.008981 sec","job_status":"done","queue_duration_s":0.418056,"scheduling_latency_s":0.41807,"redis_calls":13,"redis_duration_s":0.001546,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008981,"completed_at":"2026-09-06T16:29:03.431Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1587gitlab # [ 173.604645] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d834305add68921191766fbc","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d834305add68921191766fbc: start","job_status":"start","queue_duration_s":0.42789,"scheduling_latency_s":0.427903}1588gitlab # [ 173.605432] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d834305add68921191766fbc","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d834305add68921191766fbc: done: 0.00894 sec","job_status":"done","queue_duration_s":0.42789,"scheduling_latency_s":0.427903,"redis_calls":13,"redis_duration_s":0.001548,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001045,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005654,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00894,"completed_at":"2026-09-06T16:29:03.441Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1589gitlab # [ 173.653520] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.442Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"933bcb2765a050c7ab2ff43b","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-933bcb2765a050c7ab2ff43b: start","job_status":"start","queue_duration_s":0.437786,"scheduling_latency_s":0.4378}1590gitlab # [ 173.654450] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.451Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"933bcb2765a050c7ab2ff43b","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-933bcb2765a050c7ab2ff43b: done: 0.009072 sec","job_status":"done","queue_duration_s":0.437786,"scheduling_latency_s":0.4378,"redis_calls":13,"redis_duration_s":0.00162,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001076,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005722,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009072,"completed_at":"2026-09-06T16:29:03.451Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1591gitlab # [ 173.666146] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5e9eef8a0b0544882187759","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5e9eef8a0b0544882187759: start","job_status":"start","queue_duration_s":0.447773,"scheduling_latency_s":0.447786}1592gitlab # [ 173.677656] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.461Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5e9eef8a0b0544882187759","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5e9eef8a0b0544882187759: done: 0.009177 sec","job_status":"done","queue_duration_s":0.447773,"scheduling_latency_s":0.447786,"redis_calls":13,"redis_duration_s":0.001616,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001083,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005808,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009177,"completed_at":"2026-09-06T16:29:03.461Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1593gitlab # [ 173.722475] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.462Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1cda51275258721f6c71847c","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1cda51275258721f6c71847c: start","job_status":"start","queue_duration_s":0.457843,"scheduling_latency_s":0.457858}1594gitlab # [ 173.723761] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.471Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1cda51275258721f6c71847c","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1cda51275258721f6c71847c: done: 0.009053 sec","job_status":"done","queue_duration_s":0.457843,"scheduling_latency_s":0.457858,"redis_calls":13,"redis_duration_s":0.001586,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001052,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005743,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009053,"completed_at":"2026-09-06T16:29:03.471Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1595gitlab # [ 173.759586] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.472Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f70e81ed4bcd285fa07deefa","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f70e81ed4bcd285fa07deefa: start","job_status":"start","queue_duration_s":0.467742,"scheduling_latency_s":0.467756}1596gitlab # [ 173.804606] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.481Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f70e81ed4bcd285fa07deefa","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f70e81ed4bcd285fa07deefa: done: 0.009364 sec","job_status":"done","queue_duration_s":0.467742,"scheduling_latency_s":0.467756,"redis_calls":13,"redis_duration_s":0.0015899999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005873,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009364,"completed_at":"2026-09-06T16:29:03.481Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1597gitlab # [ 173.805766] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.482Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff7451825f57ab9d9be29927","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff7451825f57ab9d9be29927: start","job_status":"start","queue_duration_s":0.477951,"scheduling_latency_s":0.477965}1598gitlab # [ 173.817055] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff7451825f57ab9d9be29927","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff7451825f57ab9d9be29927: done: 0.00911 sec","job_status":"done","queue_duration_s":0.477951,"scheduling_latency_s":0.477965,"redis_calls":13,"redis_duration_s":0.001571,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001026,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005802,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00911,"completed_at":"2026-09-06T16:29:03.491Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1599gitlab # [ 173.858757] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.492Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e556815c6af2dcbfba996eb","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e556815c6af2dcbfba996eb: start","job_status":"start","queue_duration_s":0.487907,"scheduling_latency_s":0.487921}1600gitlab # [ 173.863483] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.501Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e556815c6af2dcbfba996eb","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e556815c6af2dcbfba996eb: done: 0.009013 sec","job_status":"done","queue_duration_s":0.487907,"scheduling_latency_s":0.487921,"redis_calls":13,"redis_duration_s":0.001549,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001041,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00573,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009013,"completed_at":"2026-09-06T16:29:03.501Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1601gitlab # [ 173.918705] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.502Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ecb508ad69d263f0f2021b26","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ecb508ad69d263f0f2021b26: start","job_status":"start","queue_duration_s":0.49778,"scheduling_latency_s":0.497794}1602gitlab # [ 173.919612] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.511Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ecb508ad69d263f0f2021b26","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ecb508ad69d263f0f2021b26: done: 0.009004 sec","job_status":"done","queue_duration_s":0.49778,"scheduling_latency_s":0.497794,"redis_calls":13,"redis_duration_s":0.001639,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005722,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009004,"completed_at":"2026-09-06T16:29:03.511Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1603gitlab # [ 173.959920] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.511Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b9a9344cacb88fe58a5568f","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b9a9344cacb88fe58a5568f: start","job_status":"start","queue_duration_s":0.507649,"scheduling_latency_s":0.507663}1604gitlab # [ 173.960725] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.521Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b9a9344cacb88fe58a5568f","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b9a9344cacb88fe58a5568f: done: 0.009258 sec","job_status":"done","queue_duration_s":0.507649,"scheduling_latency_s":0.507663,"redis_calls":13,"redis_duration_s":0.001701,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001183,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005854,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009258,"completed_at":"2026-09-06T16:29:03.521Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1605gitlab # [ 174.001727] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.522Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d498497c20aa536eccf6030e","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d498497c20aa536eccf6030e: start","job_status":"start","queue_duration_s":0.517772,"scheduling_latency_s":0.517785}1606gitlab # [ 174.002561] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.531Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d498497c20aa536eccf6030e","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d498497c20aa536eccf6030e: done: 0.009242 sec","job_status":"done","queue_duration_s":0.517772,"scheduling_latency_s":0.517785,"redis_calls":13,"redis_duration_s":0.0016710000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001136,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005881,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009242,"completed_at":"2026-09-06T16:29:03.531Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1607gitlab # [ 174.042947] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.532Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"137d8f75d8e127bf9352a5c3","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-137d8f75d8e127bf9352a5c3: start","job_status":"start","queue_duration_s":0.527895,"scheduling_latency_s":0.527909}1608gitlab # [ 174.084734] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.541Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"137d8f75d8e127bf9352a5c3","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-137d8f75d8e127bf9352a5c3: done: 0.009237 sec","job_status":"done","queue_duration_s":0.527895,"scheduling_latency_s":0.527909,"redis_calls":13,"redis_duration_s":0.001584,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001067,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009237,"completed_at":"2026-09-06T16:29:03.541Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1609gitlab # [ 174.085445] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.542Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb82addf95e66a885cdb5d9b","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb82addf95e66a885cdb5d9b: start","job_status":"start","queue_duration_s":0.537992,"scheduling_latency_s":0.538006}1610gitlab # [ 174.129773] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb82addf95e66a885cdb5d9b","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb82addf95e66a885cdb5d9b: done: 0.009231 sec","job_status":"done","queue_duration_s":0.537992,"scheduling_latency_s":0.538006,"redis_calls":13,"redis_duration_s":0.0015999999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001086,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005891,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009231,"completed_at":"2026-09-06T16:29:03.551Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1611gitlab # [ 174.131074] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.552Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f802c7e78df99d3716eeef2","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f802c7e78df99d3716eeef2: start","job_status":"start","queue_duration_s":0.548113,"scheduling_latency_s":0.548126}1612gitlab # [ 174.139787] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.561Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f802c7e78df99d3716eeef2","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f802c7e78df99d3716eeef2: done: 0.008954 sec","job_status":"done","queue_duration_s":0.548113,"scheduling_latency_s":0.548126,"redis_calls":13,"redis_duration_s":0.001566,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001055,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005679,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008954,"completed_at":"2026-09-06T16:29:03.561Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1613gitlab # [ 174.211970] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4389b4ad2cf55df04a28fbf3","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4389b4ad2cf55df04a28fbf3: start","job_status":"start","queue_duration_s":0.557904,"scheduling_latency_s":0.557917}1614gitlab # [ 174.212602] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.571Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4389b4ad2cf55df04a28fbf3","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4389b4ad2cf55df04a28fbf3: done: 0.009105 sec","job_status":"done","queue_duration_s":0.557904,"scheduling_latency_s":0.557917,"redis_calls":13,"redis_duration_s":0.001628,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001065,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005806,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009105,"completed_at":"2026-09-06T16:29:03.571Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1615gitlab # [ 174.259611] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.572Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e22962380d8644b7195c3dd","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e22962380d8644b7195c3dd: start","job_status":"start","queue_duration_s":0.567869,"scheduling_latency_s":0.567883}1616gitlab # [ 174.260567] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.581Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e22962380d8644b7195c3dd","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e22962380d8644b7195c3dd: done: 0.009308 sec","job_status":"done","queue_duration_s":0.567869,"scheduling_latency_s":0.567883,"redis_calls":13,"redis_duration_s":0.001797,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001281,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005854,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009308,"completed_at":"2026-09-06T16:29:03.581Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1617gitlab # [ 174.267774] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31e544a768b571376b1d3bd6","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31e544a768b571376b1d3bd6: start","job_status":"start","queue_duration_s":0.578019,"scheduling_latency_s":0.578034}1618gitlab # [ 174.279208] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.591Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31e544a768b571376b1d3bd6","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31e544a768b571376b1d3bd6: done: 0.008963 sec","job_status":"done","queue_duration_s":0.578019,"scheduling_latency_s":0.578034,"redis_calls":13,"redis_duration_s":0.0015730000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001066,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005664,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008963,"completed_at":"2026-09-06T16:29:03.591Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1619gitlab # [ 174.320670] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.592Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8153265288cdda524b744cb3","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8153265288cdda524b744cb3: start","job_status":"start","queue_duration_s":0.587838,"scheduling_latency_s":0.587852}1620gitlab # [ 174.326513] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8153265288cdda524b744cb3","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8153265288cdda524b744cb3: done: 0.009104 sec","job_status":"done","queue_duration_s":0.587838,"scheduling_latency_s":0.587852,"redis_calls":13,"redis_duration_s":0.001558,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001014,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009104,"completed_at":"2026-09-06T16:29:03.601Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1621gitlab # [ 174.371879] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.602Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"317f4ac3c9a0221c07c00296","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-317f4ac3c9a0221c07c00296: start","job_status":"start","queue_duration_s":0.597946,"scheduling_latency_s":0.597959}1622gitlab # [ 174.372653] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.611Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"317f4ac3c9a0221c07c00296","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-317f4ac3c9a0221c07c00296: done: 0.009108 sec","job_status":"done","queue_duration_s":0.597946,"scheduling_latency_s":0.597959,"redis_calls":13,"redis_duration_s":0.0015940000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001079,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005768,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009108,"completed_at":"2026-09-06T16:29:03.611Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1623gitlab # [ 174.422061] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.612Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"057fe1e51d40a8efe7d44226","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-057fe1e51d40a8efe7d44226: start","job_status":"start","queue_duration_s":0.607897,"scheduling_latency_s":0.60791}1624gitlab # [ 174.422747] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.621Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"057fe1e51d40a8efe7d44226","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-057fe1e51d40a8efe7d44226: done: 0.009161 sec","job_status":"done","queue_duration_s":0.607897,"scheduling_latency_s":0.60791,"redis_calls":13,"redis_duration_s":0.0015860000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001082,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005822,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009161,"completed_at":"2026-09-06T16:29:03.621Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1625gitlab # [ 174.452617] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.622Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7732ef6241028341aebaf32","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7732ef6241028341aebaf32: start","job_status":"start","queue_duration_s":0.617911,"scheduling_latency_s":0.617925}1626gitlab # [ 174.463647] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.631Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7732ef6241028341aebaf32","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7732ef6241028341aebaf32: done: 0.009174 sec","job_status":"done","queue_duration_s":0.617911,"scheduling_latency_s":0.617925,"redis_calls":13,"redis_duration_s":0.001557,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001041,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009174,"completed_at":"2026-09-06T16:29:03.631Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1627gitlab # [ 174.507692] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.632Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"59dae63012aced34db3353ae","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-59dae63012aced34db3353ae: start","job_status":"start","queue_duration_s":0.627928,"scheduling_latency_s":0.627943}1628gitlab # [ 174.508570] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:03.641Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"59dae63012aced34db3353ae","created_at":"2026-09-06T16:29:02.962Z","trace_propagation_headers":{"sentry-trace":"a5abdad12adc4f60aad0e7d73e136ee9-72a59178b8374e15","baggage":"sentry-trace_id=a5abdad12adc4f60aad0e7d73e136ee9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"11de69bf8abc471615322b56e1381bbd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:03.004Z","job_size_bytes":2,"pid":1641,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-59dae63012aced34db3353ae: done: 0.008996 sec","job_status":"done","queue_duration_s":0.627928,"scheduling_latency_s":0.627943,"redis_calls":13,"redis_duration_s":0.0015530000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001047,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005693,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008996,"completed_at":"2026-09-06T16:29:03.641Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1629gitlab # [ 174.728425] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 2325755186 wd_nsec: 23257551901630gitlab # [ 186.360995] postgres[2432]: [2432] ERROR: relation "table_without_models" does not exist1631gitlab # [ 186.363119] postgres[2432]: [2432] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1632gitlab # [ 189.319387] runuser[2233]: pam_unix(runuser:session): session closed for user gitlab1633gitlab # [ 189.322308] setup-gitlab-start[2209]: + systemctl restart gitlab-sidekiq.service1634gitlab # [ 189.349237] systemd[1]: Stopping gitlab-sidekiq.service...1635gitlab # [ 189.357875] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:20.589Z","message":"Shutting down"}1636gitlab # [ 189.359437] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:20.589Z","message":"Scheduler exiting..."}1637gitlab # [ 189.360434] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:20.590Z","message":"Terminating quiet threads for default capsule"}1638gitlab # [ 189.361860] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:20.590Z","message":"Scheduler exiting..."}1639gitlab # [ 189.866579] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:21.098Z","message":"Pausing to allow jobs to finish..."}1640gitlab # [ 192.871792] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:24.103Z","message":"Bye!"}1641gitlab # [ 192.903556] sidekiq-cluster[1641]: {"severity":"INFO","time":"2026-09-06T16:29:24.135Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1641,"worker_id":"sidekiq_0","memwd_rss_bytes":940695552,"retry":0}1642gitlab # [ 194.163254] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1643gitlab # [ 194.165200] systemd[1]: Stopped gitlab-sidekiq.service.1644gitlab # [ 194.166942] systemd[1]: gitlab-sidekiq.service: Consumed 1min 805ms CPU time over 1min 58.173s wall clock time, 962.2M memory peak, 4.6M read from disk.1645gitlab # [ 194.171962] systemd[1]: Started gitlab-sidekiq.service.1646gitlab # [ 194.175260] setup-gitlab-start[2209]: + mkdir -p /var/lib/secrets1647gitlab # [ 194.178499] setup-gitlab-start[2209]: + echo nixbot-test-token-of-20-chars1648gitlab # [ 194.180312] setup-gitlab-start[2209]: + chmod 644 /var/lib/secrets/gitlab-token1649gitlab # [ 194.182208] setup-gitlab-start[2209]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1650gitlab # [ 194.482576] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:25 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 2681651gitlab # [ 194.942502] gitaly[921]: time="2026-09-06T16:29:26.174Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRV8B1G0429AB868G726S4 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:29:36.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:26.173" pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1652gitlab # [ 194.944515] gitaly[921]: time="2026-09-06T16:29:26.174Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRV8B1G0429AB868G726S4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:29:36.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:26.173" grpc.time_ms=0.471 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1653gitlab # [ 195.032168] gitaly[921]: time="2026-09-06T16:29:26.264Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRV8B1G0429AB868G726S4 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:29:36.093" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:26.264" pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1654gitlab # [ 195.034164] gitaly[921]: time="2026-09-06T16:29:26.264Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRV8B1G0429AB868G726S4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:29:36.093" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:26.264" grpc.time_ms=0.355 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1655gitlab # [ 195.140857] gitaly[921]: time="2026-09-06T16:29:26.373Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRV8B1G0429AB868G726S4 diskcache=eecf990b-a39e-43f4-997a-cc2786bedbee grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-06T16:29:36.147" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:26.318" pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1656gitlab # [ 195.163083] gitaly[921]: time="2026-09-06T16:29:26.373Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1571 command.oublock=32 command.real_time_ms=26 command.spawn_token_fork_ms=1 command.system_time_ms=15 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRV8B1G0429AB868G726S4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-06T16:29:36.147" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:26.318" grpc.time_ms=55.071 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1657gitlab # [ 195.188353] gitaly[921]: time="2026-09-06T16:29:26.386Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRV8B1G0429AB868G726S4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:29:36.215" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:26.386" pid=921 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1658gitlab # [ 195.188913] gitaly[921]: time="2026-09-06T16:29:26.387Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRV8B1G0429AB868G726S4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:29:36.215" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:26.386" grpc.time_ms=0.698 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1659gitlab # [ 195.189651] gitaly[921]: time="2026-09-06T16:29:26.403Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRV8B1G0429AB868G726S4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:29:36.223" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:26.393" grpc.time_ms=9.9 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1660gitlab # [ 195.230809] gitaly[921]: time="2026-09-06T16:29:26.463Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRV8B1G0429AB868G726S4 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:29:36.292" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:26.462" pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1661gitlab # [ 195.240446] gitaly[921]: time="2026-09-06T16:29:26.463Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRV8B1G0429AB868G726S4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:29:36.292" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:26.462" grpc.time_ms=0.249 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1662gitlab # [ 195.277212] gitaly[921]: time="2026-09-06T16:29:26.509Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRV8B1G0429AB868G726S4 diskcache=24ceaa84-dd69-43f7-a033-e753dd64f538 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-06T16:29:36.295" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:26.465" pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1663gitlab # [ 195.286672] gitaly[921]: time="2026-09-06T16:29:26.519Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1578 command.oublock=32 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRV8B1G0429AB868G726S4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-06T16:29:36.295" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:26.465" grpc.time_ms=44.033 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1664gitlab # [ 195.304732] gitaly[921]: time="2026-09-06T16:29:26.537Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRV8B1G0429AB868G726S4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:29:36.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:26.536" pid=921 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1665gitlab # [ 195.315173] gitaly[921]: time="2026-09-06T16:29:26.537Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRV8B1G0429AB868G726S4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:29:36.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:26.536" grpc.time_ms=0.406 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1666gitlab # [ 195.325737] gitaly[921]: time="2026-09-06T16:29:26.557Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRV8B1G0429AB868G726S4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:29:36.016" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:26.547" grpc.time_ms=10.437 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1667gitlab # [ 195.373541] gitaly[921]: time="2026-09-06T16:29:26.605Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRV8B1G0429AB868G726S4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:29:36.074" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:26.605" grpc.time_ms=0.316 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1668gitlab # [ 195.385708] gitaly[921]: time="2026-09-06T16:29:26.618Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRV8B1G0429AB868G726S4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-06T16:29:36.086" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:26.617" grpc.time_ms=0.445 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1669gitlab # [ 195.907115] gitaly[921]: time="2026-09-06T16:29:27.139Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRV8B1G0429AB868G726S4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:29:37.008" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:27.128" grpc.time_ms=10.474 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1670gitlab # [ 196.150925] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:27 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 16521671gitlab # [ 196.155416] setup-gitlab-start[2491]: {"id":1,"message":null,"error":null}1672gitlab # [ 197.337665] sidekiq-cluster[2479]: {"severity":"INFO","time":"2026-09-06T16:29:28.552Z","message":"Starting cluster with 1 processes"}1673gitlab # [ 201.249867] gitaly[921]: time="2026-09-06T16:29:32.481Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVEVPEY9948C5N15K663Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:29:42.300" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:32.470" grpc.time_ms=10.478 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1674gitlab # [ 201.335893] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:32 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 1601675gitlab # [ 201.341851] setup-gitlab-start[2209]: + mkdir -p /tmp/test-flake1676gitlab # [ 201.344342] setup-gitlab-start[2209]: + cd /tmp/test-flake1677gitlab # [ 201.346101] setup-gitlab-start[2209]: + git init -b master1678gitlab # [ 201.363890] setup-gitlab-start[2563]: Initialized empty Git repository in /tmp/test-flake/.git/1679gitlab # [ 201.366987] setup-gitlab-start[2209]: + git config user.name test1680gitlab # [ 201.369100] setup-gitlab-start[2209]: + git config user.email test@example.com1681gitlab # [ 201.373451] setup-gitlab-start[2209]: + cat1682gitlab # [ 201.376379] setup-gitlab-start[2209]: + git add flake.nix1683gitlab # [ 201.380818] setup-gitlab-start[2209]: + git commit -m 'initial commit'1684gitlab # [ 201.400546] setup-gitlab-start[2568]: [master (root-commit) 12bf65e] initial commit1685gitlab # [ 201.403165] setup-gitlab-start[2568]: 1 file changed, 10 insertions(+)1686gitlab # [ 201.403826] setup-gitlab-start[2568]: create mode 100644 flake.nix1687gitlab # [ 201.404944] setup-gitlab-start[2209]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1688gitlab # [ 201.406110] setup-gitlab-start[2209]: + git push -u origin master1689gitlab # [ 201.461415] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:32 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 391690gitlab # [ 201.563137] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:32 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 951691gitlab # [ 201.563875] gitaly[921]: time="2026-09-06T16:29:32.794Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF4V8XWWCWJ97E2RT9QK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:32.783" grpc.time_ms=11.061 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1692gitlab # [ 201.764817] gitaly[921]: time="2026-09-06T16:29:32.997Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:29:42.009" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:32.989" grpc.time_ms=7.39 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1693gitlab # [ 201.787620] gitaly[921]: time="2026-09-06T16:29:33.020Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-06T16:30:03.014" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-06T16:29:33.014" grpc.time_ms=5.196 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1694gitlab # [ 201.806569] gitaly[921]: time="2026-09-06T16:29:33.038Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ duration_ms=97ns method=POST pid=921 status=200 url="http://unix/api/v4/internal/allowed"1695gitlab # [ 201.828937] gitaly[921]: time="2026-09-06T16:29:33.061Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ duration_ms=21ns method=POST pid=921 status=200 url="http://unix/api/v4/internal/pre_receive"1696gitlab # [ 201.829547] gitaly[921]: time="2026-09-06T16:29:33.061Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:29:32.940" grpc.time_ms=121.283 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1697gitlab # [ 201.862437] gitaly[921]: time="2026-09-06T16:29:33.094Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:29:33.094" grpc.time_ms=0.502 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1698gitlab # [ 201.894422] gitaly[921]: time="2026-09-06T16:29:33.126Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:29:33.126" grpc.time_ms=0.483 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1699gitlab # [ 201.922582] gitaly[921]: time="2026-09-06T16:29:33.154Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:29:33.154" grpc.time_ms=0.664 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1700gitlab # [ 201.994892] gitaly[921]: time="2026-09-06T16:29:33.226Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:29:43.050" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:33.220" grpc.time_ms=5.579 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1701gitlab # [ 202.031142] gitaly[921]: time="2026-09-06T16:29:33.263Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1568 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-06T16:29:43.080" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:29:33.251" grpc.time_ms=12.28 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1702gitlab # [ 202.051471] gitaly[921]: time="2026-09-06T16:29:33.283Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ duration_ms=100ns method=POST pid=921 status=200 url="http://unix/api/v4/internal/post_receive"1703gitlab # [ 202.054630] gitaly[921]: time="2026-09-06T16:29:33.287Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:29:33.182" grpc.time_ms=104.507 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1704gitlab # [ 202.072767] gitaly[921]: time="2026-09-06T16:29:33.305Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ diskcache=b336e1f5-fa6e-4796-8277-b08b83744ee0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:32.893" pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1705gitlab # [ 202.073457] gitaly[921]: time="2026-09-06T16:29:33.305Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=148 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=901120 command.minflt=13586 command.oublock=32 command.real_time_ms=402 command.spawn_token_fork_ms=0 command.system_time_ms=95 command.user_time_ms=53 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:32.893" grpc.time_ms=411.496 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1706gitlab # [ 202.074101] gitlab-workhorse[826]: time="2026-09-06T16:29:33Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521707gitlab # [ 202.074574] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:33 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 4711708gitlab # [ 202.097580] setup-gitlab-start[2572]: To http://gitlab/root/test-flake.git1709gitlab # [ 202.098505] setup-gitlab-start[2572]: * [new branch] master -> master1710gitlab # [ 202.102205] setup-gitlab-start[2572]: branch 'master' set up to track 'origin/master'.1711gitlab # [ 202.104704] systemd[1]: Finished setup-gitlab.service.1712gitlab # [ 202.107349] systemd[1]: Starting nixbot CI service...1713gitlab: (finished: waiting for unit setup-gitlab.service, in 203.05 seconds)1714gitlab: waiting for unit nixbot.service1715gitlab # [ 203.172731] nixbot[2644]: {"ts": "2026-09-06T16:29:34.404Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1716gitlab # [ 203.203548] nixbot[2644]: {"ts": "2026-09-06T16:29:34.435Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1717gitlab # [ 203.231511] nixbot[2644]: {"ts": "2026-09-06T16:29:34.463Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1718gitlab # [ 203.238658] nixbot[2644]: {"ts": "2026-09-06T16:29:34.470Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1719gitlab # [ 203.243087] nixbot[2644]: {"ts": "2026-09-06T16:29:34.475Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1720gitlab # [ 203.252478] nixbot[2644]: {"ts": "2026-09-06T16:29:34.484Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1721gitlab # [ 203.258445] nixbot[2644]: {"ts": "2026-09-06T16:29:34.490Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1722gitlab # [ 203.262494] nixbot[2644]: {"ts": "2026-09-06T16:29:34.494Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1723gitlab # [ 203.270212] nixbot[2644]: {"ts": "2026-09-06T16:29:34.502Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1724gitlab # [ 203.274614] nixbot[2644]: {"ts": "2026-09-06T16:29:34.506Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1725gitlab # [ 203.282941] nixbot[2644]: {"ts": "2026-09-06T16:29:34.515Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1726gitlab # [ 203.292848] nixbot[2644]: {"ts": "2026-09-06T16:29:34.525Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1727gitlab # [ 203.298813] nixbot[2644]: {"ts": "2026-09-06T16:29:34.531Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1728gitlab # [ 203.303807] nixbot[2644]: {"ts": "2026-09-06T16:29:34.536Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1729gitlab # [ 203.308956] nixbot[2644]: {"ts": "2026-09-06T16:29:34.541Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1730gitlab # [ 203.313379] nixbot[2644]: {"ts": "2026-09-06T16:29:34.545Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1731gitlab # [ 203.317426] nixbot[2644]: {"ts": "2026-09-06T16:29:34.549Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1732gitlab # [ 203.321347] nixbot[2644]: {"ts": "2026-09-06T16:29:34.553Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1733gitlab # [ 203.326988] nixbot[2644]: {"ts": "2026-09-06T16:29:34.559Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1734gitlab # [ 203.331104] nixbot[2644]: {"ts": "2026-09-06T16:29:34.563Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1735gitlab # [ 203.335310] nixbot[2644]: {"ts": "2026-09-06T16:29:34.567Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1736gitlab # [ 203.341082] nixbot[2644]: {"ts": "2026-09-06T16:29:34.573Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1737gitlab # [ 203.345080] nixbot[2644]: {"ts": "2026-09-06T16:29:34.577Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1738gitlab # [ 203.348498] nixbot[2644]: {"ts": "2026-09-06T16:29:34.580Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1739gitlab # [ 203.352340] nixbot[2644]: {"ts": "2026-09-06T16:29:34.584Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1740gitlab # [ 203.356794] nixbot[2644]: {"ts": "2026-09-06T16:29:34.589Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1741gitlab # [ 203.360914] nixbot[2644]: {"ts": "2026-09-06T16:29:34.593Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1742gitlab # [ 203.364740] nixbot[2644]: {"ts": "2026-09-06T16:29:34.597Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1743gitlab # [ 203.368658] nixbot[2644]: {"ts": "2026-09-06T16:29:34.600Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1744gitlab # [ 203.382363] nixbot[2644]: {"ts": "2026-09-06T16:29:34.614Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1745gitlab # [ 203.387733] nixbot[2644]: {"ts": "2026-09-06T16:29:34.620Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1746gitlab # [ 203.391626] nixbot[2644]: {"ts": "2026-09-06T16:29:34.623Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1747gitlab # [ 203.395596] nixbot[2644]: {"ts": "2026-09-06T16:29:34.627Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"}1748gitlab # [ 203.716331] nixbot[2644]: INFO: Started server process [2644]1749gitlab # [ 203.718340] nixbot[2644]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1750gitlab # [ 203.738652] nixbot[2644]: INFO: ::ffff:127.0.0.1:56412 - "GET /health HTTP/1.1" 200 OK1751gitlab # [ 203.742412] systemd[1]: Started nixbot CI service.1752gitlab # [ 203.745124] systemd[1]: Reached target Multi-User System.1753gitlab # [ 203.747471] systemd[1]: Startup finished in 1.211s (kernel) + 3.646s (initrd) + 3min 18.889s (userspace) = 3min 23.746s.1754gitlab # [ 203.952797] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:35 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 2171755gitlab # [ 203.954630] nixbot[2644]: {"ts": "2026-09-06T16:29:35.186Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1756gitlab # [ 203.979727] nixbot[2644]: {"ts": "2026-09-06T16:29:35.211Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1757gitlab # [ 204.065678] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:35 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 691758gitlab # [ 204.067328] nixbot[2644]: {"ts": "2026-09-06T16:29:35.298Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1759gitlab # [ 204.068402] nixbot[2644]: {"ts": "2026-09-06T16:29:35.299Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1760gitlab # [ 204.290171] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:35 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 2091761gitlab # [ 204.295872] nixbot[2644]: {"ts": "2026-09-06T16:29:35.523Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1762gitlab # [ 204.476174] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:35 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 1731763gitlab # [ 204.481584] nixbot[2644]: {"ts": "2026-09-06T16:29:35.706Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1764gitlab: (finished: waiting for unit nixbot.service, in 2.07 seconds)1765gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1766gitlab # [ 204.529154] nixbot[2644]: INFO: ::ffff:127.0.0.1:53230 - "GET /health HTTP/1.1" 200 OK1767gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.02 seconds)1768subtest: project discovered and webhook registered1769gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1770gitlab # [ 204.596785] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:35 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 451771gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.07 seconds)1772(finished: subtest: project discovered and webhook registered, in 0.07 seconds)1773subtest: push delivers the webhook1774gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1775gitlab # [ 204.995155] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:36 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 4971776gitlab # [ 205.000304] nixbot[2644]: {"ts": "2026-09-06T16:29:36.227Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1777gitlab # [ 205.071774] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:36 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 731778gitlab # [ 205.076544] nixbot[2644]: {"ts": "2026-09-06T16:29:36.304Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1779gitlab # [ 205.162320] gitaly[921]: time="2026-09-06T16:29:36.394Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVJNRHVZTJHFMKVK9J9EB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-06T16:30:06.078" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:29:36.378" grpc.time_ms=15.28 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1780gitlab # [ 205.225785] gitaly[921]: time="2026-09-06T16:29:36.458Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVJNRHVZTJHFMKVK9J9EB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:30:06.156" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:29:36.456" grpc.time_ms=1.108 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1781gitlab # [ 205.240497] gitaly[921]: time="2026-09-06T16:29:36.467Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVJNRHVZTJHFMKVK9J9EB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-06T16:29:46.010" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:29:36.460" grpc.time_ms=5.882 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1782gitlab # [ 205.296910] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:36 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 2161783gitlab # [ 205.300914] nixbot[2644]: {"ts": "2026-09-06T16:29:36.530Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1784gitlab # [ 205.380454] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:36 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 791785gitlab # [ 205.383446] nixbot[2644]: {"ts": "2026-09-06T16:29:36.614Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1786gitlab # [ 205.386590] nixbot[2644]: {"ts": "2026-09-06T16:29:36.618Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1787gitlab # [ 205.472961] gitaly[921]: time="2026-09-06T16:29:36.705Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVK0F3XCQA3723KZNCNPB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:36.697" grpc.time_ms=8.025 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc1788gitlab # [ 205.489337] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:36 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 651789gitlab # [ 205.597562] gitaly[921]: time="2026-09-06T16:29:36.829Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVK2J8NA7ZCAAAC1ZHRSS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:36.818" pid=921 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1790gitlab # [ 205.607024] gitaly[921]: time="2026-09-06T16:29:36.830Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVK2J8NA7ZCAAAC1ZHRSS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:36.818" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc1791gitlab # [ 205.618901] gitaly[921]: time="2026-09-06T16:29:36.830Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVK2J8NA7ZCAAAC1ZHRSS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:36.818" grpc.time_ms=11.623 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc1792gitlab # [ 205.637288] gitlab-workhorse[826]: time="2026-09-06T16:29:36Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131793gitlab # [ 205.639292] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:36 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 1421794gitlab # [ 205.720062] gitaly[921]: time="2026-09-06T16:29:36.952Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVK74JAQDPC3FFEMSMK10 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:36.938" pid=921 remote_ip="2001:db8:1::1" request_sha=ae719ac978d26ef2e8b515f856004dd7de584cae response_bytes=223 span.kind=server system=grpc1795gitlab # [ 205.729601] gitaly[921]: time="2026-09-06T16:29:36.952Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1195 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVK74JAQDPC3FFEMSMK10 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:36.938" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc1796gitlab # [ 205.741506] gitlab-workhorse[826]: time="2026-09-06T16:29:36Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2231797gitlab # [ 205.743293] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:36 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 841798gitlab # [ 205.745126] gitaly[921]: time="2026-09-06T16:29:36.952Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1195 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVK74JAQDPC3FFEMSMK10 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:36.938" grpc.time_ms=14.703 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc1799gitlab # [ 205.860657] gitaly[921]: time="2026-09-06T16:29:37.092Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVKC32KK6G5EZRNZNRY8S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:37.082" grpc.time_ms=9.68 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc1800gitlab # [ 205.888575] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:37 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 651801gitlab # [ 205.937320] gitaly[921]: time="2026-09-06T16:29:37.169Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVKE6ACV3QQY640BHZFFP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:37.162" pid=921 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1802gitlab # [ 205.958416] gitaly[921]: time="2026-09-06T16:29:37.172Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVKE6ACV3QQY640BHZFFP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:37.162" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc1803gitlab # [ 205.975934] gitlab-workhorse[826]: time="2026-09-06T16:29:37Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671804gitlab # [ 205.976506] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:37 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 781805gitlab # [ 205.977049] gitaly[921]: time="2026-09-06T16:29:37.172Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVKE6ACV3QQY640BHZFFP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:37.162" grpc.time_ms=10.339 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc1806gitlab # [ 206.056570] gitaly[921]: time="2026-09-06T16:29:37.288Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVKJX47D6MNBCXC2WVM3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:37.281" grpc.time_ms=7.638 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc1807gitlab # [ 206.078647] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:37 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 641808gitlab # [ 206.148687] gitaly[921]: time="2026-09-06T16:29:37.381Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVKN2EK14CFR0934BTFDM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:37.365" pid=921 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1809gitlab # [ 206.149903] gitaly[921]: time="2026-09-06T16:29:37.381Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1165 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVKN2EK14CFR0934BTFDM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:37.365" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc1810gitlab # [ 206.150946] gitaly[921]: time="2026-09-06T16:29:37.381Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1165 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVKN2EK14CFR0934BTFDM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:37.365" grpc.time_ms=15.719 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc1811gitlab # [ 206.152092] gitlab-workhorse[826]: time="2026-09-06T16:29:37Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091812gitlab # [ 206.152773] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:37 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 681813gitlab # [ 206.292107] gitaly[921]: time="2026-09-06T16:29:37.524Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:30:07.522" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:29:37.523" grpc.time_ms=0.704 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1814gitlab # [ 206.326152] gitaly[921]: time="2026-09-06T16:29:37.558Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-06T16:30:07.551" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:29:37.551" grpc.time_ms=6.324 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1815gitlab # [ 206.352202] gitaly[921]: time="2026-09-06T16:29:37.584Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:29:47.043" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:29:37.573" grpc.time_ms=10.472 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1816gitlab # [ 206.365808] gitaly[921]: time="2026-09-06T16:29:37.598Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:29:47.063" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:29:37.593" grpc.time_ms=4.712 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1817gitlab # [ 206.431121] gitaly[921]: time="2026-09-06T16:29:37.663Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-06T16:29:47.132" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:37.662" grpc.time_ms=0.402 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1818gitlab # [ 206.459772] gitaly[921]: time="2026-09-06T16:29:37.692Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-06T16:29:47.154" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:29:37.685" grpc.time_ms=6.717 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1819gitlab # [ 206.519330] gitaly[921]: time="2026-09-06T16:29:37.751Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:29:37.751" grpc.time_ms=0.426 pid=921 span.kind=server system=grpc1820gitlab # [ 206.553081] gitaly[921]: time="2026-09-06T16:29:37.785Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:29:37.784" grpc.time_ms=0.373 pid=921 span.kind=server system=grpc1821gitlab # [ 206.570380] gitaly[921]: time="2026-09-06T16:29:37.802Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 diskcache=1ce38e3f-a7cb-4220-b9a6-4936af51879e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:29:47.174" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:37.704" pid=921 ref=refs/keep-around/12bf65ece35b0516d0c8e2591b41125a05ab1df8 remote_ip="2001:db8:1::1" revision=12bf65ece35b0516d0c8e2591b41125a05ab1df8 span.kind=server system=grpc username=root1822gitlab # [ 206.571105] gitaly[921]: time="2026-09-06T16:29:37.803Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=79 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5509 command.oublock=8 command.real_time_ms=89 command.spawn_token_fork_ms=0 command.system_time_ms=50 command.user_time_ms=29 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:29:47.174" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:29:37.704" grpc.time_ms=97.894 pid=921 ref=refs/keep-around/12bf65ece35b0516d0c8e2591b41125a05ab1df8 remote_ip="2001:db8:1::1" revision=12bf65ece35b0516d0c8e2591b41125a05ab1df8 span.kind=server system=grpc username=root1823gitlab # [ 206.597225] gitaly[921]: time="2026-09-06T16:29:37.829Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:30:07.828" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:29:37.829" grpc.time_ms=0.303 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1824gitlab # [ 206.740102] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:37 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/12bf65ece35b0516d0c8e2591b41125a05ab1df8 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 4951825gitlab # [ 206.747496] nixbot[2644]: {"ts": "2026-09-06T16:29:37.979Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/12bf65ece35b0516d0c8e2591b41125a05ab1df8 \"HTTP/1.1 201 Created\""}1826gitlab # [ 206.764215] nixbot[2644]: {"ts": "2026-09-06T16:29:37.996Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-12bf65ece35b-0"}1827gitlab # [ 206.941358] nixbot[2644]: {"ts": "2026-09-06T16:29:38.173Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-12bf65ece35b-0", "argv0": "bwrap"}1828gitlab # [ 207.052415] systemd[1]: Started Nix Daemon.1829gitlab # [ 207.102801] nix-daemon[2820]: accepted connection from pid 2815, user nixbot1830gitlab # [ 211.714065] nix-daemon[2820]: accepted connection from pid 2863, user nixbot1831gitlab # [ 211.727668] nix-daemon[2820]: accepted connection from pid 2815, user nixbot1832gitlab # [ 216.344431] nix-daemon[2820]: accepted connection from pid 2815, user nixbot1833gitlab # [ 216.441643] nix-daemon[2820]: accepted connection from pid 2912, user nixbot1834gitlab # [ 216.497856] gitaly[921]: time="2026-09-06T16:29:47.729Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVXR8D3HMZQ4CKCB0ACEX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:30:17.727" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:29:47.728" grpc.time_ms=0.876 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1835gitlab # [ 216.537412] gitaly[921]: time="2026-09-06T16:29:47.769Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVXR8D3HMZQ4CKCB0ACEX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:29:57.227" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:29:47.758" grpc.time_ms=11.088 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1836gitlab # [ 216.576216] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:47 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/12bf65ece35b0516d0c8e2591b41125a05ab1df8 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 1501837gitlab # [ 216.577771] nixbot[2644]: {"ts": "2026-09-06T16:29:47.808Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/12bf65ece35b0516d0c8e2591b41125a05ab1df8 \"HTTP/1.1 201 Created\""}1838gitlab # [ 216.634062] gitaly[921]: time="2026-09-06T16:29:47.865Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVXX6FGQK5Q796BTQ2R1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:30:17.864" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:29:47.864" grpc.time_ms=0.612 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1839gitlab # [ 216.687341] gitaly[921]: time="2026-09-06T16:29:47.919Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVXX6FGQK5Q796BTQ2R1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:29:57.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:29:47.906" grpc.time_ms=12.27 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1840gitlab # [ 216.777718] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:48 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/12bf65ece35b0516d0c8e2591b41125a05ab1df8 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1941841gitlab # [ 216.782131] nixbot[2644]: {"ts": "2026-09-06T16:29:48.012Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/12bf65ece35b0516d0c8e2591b41125a05ab1df8 \"HTTP/1.1 201 Created\""}1842gitlab # [ 220.753035] nix-daemon[2820]: accepted connection from pid 2969, user nixbot1843gitlab # [ 220.841533] gitaly[921]: time="2026-09-06T16:29:52.073Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRW20J5PSXGS4404GZRKSC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:30:22.071" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:29:52.072" grpc.time_ms=0.694 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1844gitlab # [ 220.876325] gitaly[921]: time="2026-09-06T16:29:52.108Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1VRW20J5PSXGS4404GZRKSC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:30:02.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:29:52.095" grpc.time_ms=12.504 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1845gitlab # [ 220.944724] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:52 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/12bf65ece35b0516d0c8e2591b41125a05ab1df8 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1581846gitlab # [ 220.948090] nixbot[2644]: {"ts": "2026-09-06T16:29:52.180Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/12bf65ece35b0516d0c8e2591b41125a05ab1df8 \"HTTP/1.1 201 Created\""}1847gitlab # [ 221.266466] gitaly[921]: time="2026-09-06T16:29:52.498Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VRW2E0JTMVFPVDFRTM4JR5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:52.489" grpc.time_ms=9.282 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc1848gitlab # [ 221.292123] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:52 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 751849gitlab # [ 221.363189] gitaly[921]: time="2026-09-06T16:29:52.595Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRW2GJG6GJ6TXAVJ8GVNJP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:52.583" pid=921 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1850gitlab # [ 221.365406] gitaly[921]: time="2026-09-06T16:29:52.595Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRW2GJG6GJ6TXAVJ8GVNJP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:52.583" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc1851gitlab # [ 221.407603] gitaly[921]: time="2026-09-06T16:29:52.639Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRW2GJG6GJ6TXAVJ8GVNJP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:52.583" grpc.time_ms=12.507 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc1852gitlab # [ 221.420855] gitlab-workhorse[826]: time="2026-09-06T16:29:52Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671853gitlab # [ 221.421896] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:52 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1221854gitlab # [ 221.505333] gitaly[921]: time="2026-09-06T16:29:52.737Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRW2NJ2NGR4PSNJDCQZ1G3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:52.727" grpc.time_ms=9.614 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc1855gitlab # [ 221.526452] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:52 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 671856gitlab # [ 221.581861] gitaly[921]: time="2026-09-06T16:29:52.814Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRW2QTHGATWPAXFQ96QWY1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:52.807" pid=921 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1857gitlab # [ 221.582680] gitaly[921]: time="2026-09-06T16:29:52.814Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRW2QTHGATWPAXFQ96QWY1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:52.807" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc1858gitlab # [ 221.583749] gitlab-workhorse[826]: time="2026-09-06T16:29:52Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671859gitlab # [ 221.585248] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:29:52 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 521860gitlab # [ 221.587518] gitaly[921]: time="2026-09-06T16:29:52.814Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRW2QTHGATWPAXFQ96QWY1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:29:52.807" grpc.time_ms=6.836 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc1861gitlab # [ 223.058298] sidekiq-cluster[2536]: 2026-09-06T16:29:54.289Z pid=2536 tid=xts INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2536, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>442908672}1862gitlab # [ 223.245610] sidekiq-cluster[2536]: W, [2026-09-06T16:29:54.477788 #2536] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1863gitlab # [ 223.283510] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:29:54.515Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1864gitlab # [ 225.046314] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:29:56.277Z","message":"GitLab reliable fetch activated!"}1865gitlab # [ 225.048689] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:29:56.278Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1866gitlab # [ 245.190401] postgres[3219]: [3219] ERROR: relation "table_without_models" does not exist1867gitlab # [ 245.192976] postgres[3219]: [3219] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1868gitlab # [ 247.643742] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:18.875Z","message":"Booted Rails 7.2.3.1 application in production environment"}1869gitlab # [ 247.647963] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:18.880Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1870gitlab # [ 247.652573] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:18.880Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1871gitlab # [ 247.656567] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:18.880Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1872gitlab # [ 247.661213] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:18.884Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1873gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 43.55 seconds)1874gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1875gitlab # [ 248.243466] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:19 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 371876gitlab # [ 248.343654] gitaly[921]: time="2026-09-06T16:30:19.575Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRWWTRJMFB7NK8F782BW9E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:30:19.564" grpc.time_ms=10.684 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1877gitlab # [ 248.344801] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:19 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 201 "" "git/2.55.0" 941878gitlab # Enumerating objects: 5, done.1879gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1880gitlab # Delta compression using up to 4 threads1881gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1882gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 136.00 KiB/s, done.1883gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1884gitlab # [ 248.587764] gitaly[921]: time="2026-09-06T16:30:19.820Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=325 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-06T16:30:49.814" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-06T16:30:19.814" grpc.time_ms=5.536 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1885gitlab # [ 248.599615] gitaly[921]: time="2026-09-06T16:30:19.831Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY duration_ms=64ns method=POST pid=921 status=200 url="http://unix/api/v4/internal/allowed"1886gitlab # [ 248.617761] gitaly[921]: time="2026-09-06T16:30:19.850Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY duration_ms=18ns method=POST pid=921 status=200 url="http://unix/api/v4/internal/pre_receive"1887gitlab # [ 248.618860] gitaly[921]: time="2026-09-06T16:30:19.850Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:30:19.765" grpc.time_ms=84.744 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1888gitlab # [ 248.678594] gitaly[921]: time="2026-09-06T16:30:19.910Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:30:19.909" grpc.time_ms=0.698 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1889gitlab # [ 248.729571] gitaly[921]: time="2026-09-06T16:30:19.961Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:30:19.961" grpc.time_ms=0.528 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1890gitlab # [ 248.767744] gitaly[921]: time="2026-09-06T16:30:20.000Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:30:19.999" grpc.time_ms=0.31 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1891gitlab # [ 248.830449] gitaly[921]: time="2026-09-06T16:30:20.062Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY duration_ms=34ns method=POST pid=921 status=200 url="http://unix/api/v4/internal/post_receive"1892gitlab # [ 248.831790] gitaly[921]: time="2026-09-06T16:30:20.063Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:30:20.027" grpc.time_ms=36.177 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1893gitlab # [ 248.864450] gitaly[921]: time="2026-09-06T16:30:20.096Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY diskcache=f31fa3ce-1d8c-46dd-82ee-2ef1e542b4c1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:30:19.690" pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1894gitlab # To http://gitlab/root/test-flake.git1895gitlab # [ 248.873763] gitaly[921]: time="2026-09-06T16:30:20.096Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=203 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13636 command.oublock=32 command.real_time_ms=396 command.spawn_token_fork_ms=0 command.system_time_ms=126 command.user_time_ms=77 component=gitaly.StreamServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:30:19.690" grpc.time_ms=406 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1896gitlab # 12bf65e..d5b04fe master -> master[ 248.874837] gitlab-workhorse[826]: time="2026-09-06T16:30:20Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521897gitlab # 1898gitlab # [ 248.875976] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:20 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 4831899gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.74 seconds)1900gitlab: must succeed: git -C /tmp/test-flake rev-parse master1901gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)1902gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0'1903gitlab # [ 248.963042] nixbot[2644]: INFO: ::ffff:127.0.0.1:57056 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 HTTP/1.1" 200 OK1904gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0', in 0.07 seconds)1905gitlab # [ 249.044944] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:20.276Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b27abfbc1c797715a3c7b21b","created_at":"2026-09-06T16:29:26.414Z","trace_propagation_headers":{"sentry-trace":"53f52d1266744be3aa9c89ab785ad175-157da2d9fea44f56","baggage":"sentry-trace_id=53f52d1266744be3aa9c89ab785ad175,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRV8B1G0429AB868G726S4","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:26.439Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b27abfbc1c797715a3c7b21b: start","job_status":"start","queue_duration_s":53.836294,"scheduling_latency_s":53.836345}1906gitlab # [ 249.272952] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:20.504Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b27abfbc1c797715a3c7b21b","created_at":"2026-09-06T16:29:26.414Z","trace_propagation_headers":{"sentry-trace":"53f52d1266744be3aa9c89ab785ad175-157da2d9fea44f56","baggage":"sentry-trace_id=53f52d1266744be3aa9c89ab785ad175,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRV8B1G0429AB868G726S4","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:26.439Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b27abfbc1c797715a3c7b21b: done: 0.228289 sec","job_status":"done","queue_duration_s":53.836294,"scheduling_latency_s":53.836345,"redis_calls":24,"redis_duration_s":0.008674,"redis_read_bytes":1038,"redis_write_bytes":2604,"redis_feature_flag_calls":14,"redis_feature_flag_duration_s":0.004702,"redis_feature_flag_read_bytes":1030,"redis_feature_flag_write_bytes":1439,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00222,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001752,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.085714,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.093982615,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.228289,"completed_at":"2026-09-06T16:30:20.504Z","load_balancing_strategy":"primary","db_duration_s":0.061301,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1907gitlab # [ 249.274742] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:20.506Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"ba534bbebef6e035a43bbdf5","created_at":"2026-09-06T16:29:26.448Z","trace_propagation_headers":{"sentry-trace":"53f52d1266744be3aa9c89ab785ad175-157da2d9fea44f56","baggage":"sentry-trace_id=53f52d1266744be3aa9c89ab785ad175,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRV8B1G0429AB868G726S4","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:26.456Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-ba534bbebef6e035a43bbdf5: start","job_status":"start","queue_duration_s":54.050271,"scheduling_latency_s":54.050305}1908gitlab # [ 249.369605] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:20.601Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"ba534bbebef6e035a43bbdf5","created_at":"2026-09-06T16:29:26.448Z","trace_propagation_headers":{"sentry-trace":"53f52d1266744be3aa9c89ab785ad175-157da2d9fea44f56","baggage":"sentry-trace_id=53f52d1266744be3aa9c89ab785ad175,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRV8B1G0429AB868G726S4","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:26.456Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-ba534bbebef6e035a43bbdf5: done: 0.094844 sec","job_status":"done","queue_duration_s":54.050271,"scheduling_latency_s":54.050305,"redis_calls":14,"redis_duration_s":0.003891,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000919,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001657,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001315,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.062496,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.070354193,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.094844,"completed_at":"2026-09-06T16:30:20.601Z","load_balancing_strategy":"primary","db_duration_s":0.023453,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1909gitlab # [ 249.371301] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:20.603Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"ac0e33472bd4e6d835f6a8dd","created_at":"2026-09-06T16:29:27.025Z","trace_propagation_headers":{"sentry-trace":"53f52d1266744be3aa9c89ab785ad175-157da2d9fea44f56","baggage":"sentry-trace_id=53f52d1266744be3aa9c89ab785ad175,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRV8B1G0429AB868G726S4","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:27.031Z","job_size_bytes":5,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-ac0e33472bd4e6d835f6a8dd: start","job_status":"start","queue_duration_s":53.571695,"scheduling_latency_s":53.571709}1910gitlab # [ 249.566934] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:20.798Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"ac0e33472bd4e6d835f6a8dd","created_at":"2026-09-06T16:29:27.025Z","trace_propagation_headers":{"sentry-trace":"53f52d1266744be3aa9c89ab785ad175-157da2d9fea44f56","baggage":"sentry-trace_id=53f52d1266744be3aa9c89ab785ad175,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRV8B1G0429AB868G726S4","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:27.031Z","job_size_bytes":5,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-ac0e33472bd4e6d835f6a8dd: done: 0.195808 sec","job_status":"done","queue_duration_s":53.571695,"scheduling_latency_s":53.571709,"redis_calls":12,"redis_duration_s":0.0029519999999999998,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000955,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001591,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000406,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.061,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.061,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.116525,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000285511,"duration_s":0.195808,"completed_at":"2026-09-06T16:30:20.798Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.072234,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1911gitlab # [ 249.570550] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:20.800Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"c5a88d7867d9fead0919590d","created_at":"2026-09-06T16:29:27.052Z","trace_propagation_headers":{"sentry-trace":"53f52d1266744be3aa9c89ab785ad175-157da2d9fea44f56","baggage":"sentry-trace_id=53f52d1266744be3aa9c89ab785ad175,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRV8B1G0429AB868G726S4","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:27.056Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-c5a88d7867d9fead0919590d: start","job_status":"start","queue_duration_s":53.744216,"scheduling_latency_s":53.74423}1912gitlab # [ 249.721268] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:20.953Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"c5a88d7867d9fead0919590d","created_at":"2026-09-06T16:29:27.052Z","trace_propagation_headers":{"sentry-trace":"53f52d1266744be3aa9c89ab785ad175-157da2d9fea44f56","baggage":"sentry-trace_id=53f52d1266744be3aa9c89ab785ad175,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRV8B1G0429AB868G726S4","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:27.056Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-c5a88d7867d9fead0919590d: done: 0.152794 sec","job_status":"done","queue_duration_s":53.744216,"scheduling_latency_s":53.74423,"redis_calls":11,"redis_duration_s":0.002502,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000666,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001381,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000455,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.127,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.127,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.082036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000315403,"duration_s":0.152794,"completed_at":"2026-09-06T16:30:20.953Z","load_balancing_strategy":"primary","db_duration_s":0.070401,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1913gitlab # [ 249.723729] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:20.954Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"0f9d1b46bca8d79e23c34eb0","created_at":"2026-09-06T16:29:27.058Z","trace_propagation_headers":{"sentry-trace":"53f52d1266744be3aa9c89ab785ad175-157da2d9fea44f56","baggage":"sentry-trace_id=53f52d1266744be3aa9c89ab785ad175,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRV8B1G0429AB868G726S4","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:27.061Z","job_size_bytes":89,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-0f9d1b46bca8d79e23c34eb0: start","job_status":"start","queue_duration_s":53.893071,"scheduling_latency_s":53.893085}1914gitlab # [ 249.791785] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:20.981Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"0f9d1b46bca8d79e23c34eb0","created_at":"2026-09-06T16:29:27.058Z","trace_propagation_headers":{"sentry-trace":"53f52d1266744be3aa9c89ab785ad175-157da2d9fea44f56","baggage":"sentry-trace_id=53f52d1266744be3aa9c89ab785ad175,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRV8B1G0429AB868G726S4","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:27.061Z","job_size_bytes":89,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-0f9d1b46bca8d79e23c34eb0: done: 0.027092 sec","job_status":"done","queue_duration_s":53.893071,"scheduling_latency_s":53.893085,"redis_calls":11,"redis_duration_s":0.002456,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000671,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001303,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000482,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000319593,"duration_s":0.027092,"completed_at":"2026-09-06T16:30:20.981Z","load_balancing_strategy":"primary","db_duration_s":0.005604,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1915gitlab # [ 249.830200] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:20.982Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 12bf65ece35b0516d0c8e2591b41125a05ab1df8 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"7de861b1183ba3bb87acb865","created_at":"2026-09-06T16:29:33.209Z","trace_propagation_headers":{"sentry-trace":"0b62de9e8d014266b4c693d30fa5d97e-a1637ba99f234b54","baggage":"sentry-trace_id=0b62de9e8d014266b4c693d30fa5d97e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRVF92VENKA2GZPJSCYPBZ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:33.212Z","job_size_bytes":185,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-7de861b1183ba3bb87acb865: start","job_status":"start","queue_duration_s":47.770055,"scheduling_latency_s":47.770068}1916gitlab # [ 249.831196] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:20.994Z","message":"changes: \"0000000000000000000000000000000000000000 12bf65ece35b0516d0c8e2591b41125a05ab1df8 refs/heads/master\\n\""}1917gitlab # [ 249.845321] gitaly[921]: time="2026-09-06T16:30:21.077Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=60 command.spawn_token_fork_ms=51 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:30:31.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:21.015" grpc.time_ms=61.489 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1918gitlab # [ 249.863309] gitaly[921]: time="2026-09-06T16:30:21.095Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:30:31.075" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:30:21.085" grpc.time_ms=9.965 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1919gitlab # [ 249.925301] gitaly[921]: time="2026-09-06T16:30:21.157Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-06T16:30:31.138" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:21.148" grpc.time_ms=8.69 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1920gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0'1921gitlab # [ 249.967260] gitaly[921]: time="2026-09-06T16:30:21.199Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3933 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:30:21.199" grpc.time_ms=0.284 pid=921 span.kind=server system=grpc1922gitlab # [ 249.993397] gitaly[921]: time="2026-09-06T16:30:21.225Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3935 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:30:21.225" grpc.time_ms=0.284 pid=921 span.kind=server system=grpc1923gitlab # [ 250.010233] nixbot[2644]: INFO: ::ffff:127.0.0.1:57072 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 HTTP/1.1" 200 OK1924gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0', in 0.04 seconds)1925gitlab # [ 250.016933] gitaly[921]: time="2026-09-06T16:30:21.249Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ diskcache=8649a871-90ce-42b2-a2e8-786343a2f91b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:30:31.165" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:30:21.176" pid=921 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1926gitlab # [ 250.027369] gitaly[921]: time="2026-09-06T16:30:21.249Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=66 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4818 command.oublock=8 command.real_time_ms=63 command.spawn_token_fork_ms=0 command.system_time_ms=47 command.user_time_ms=19 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:30:31.165" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:30:21.176" grpc.time_ms=73.087 pid=921 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1927gitlab # [ 250.032918] gitaly[921]: time="2026-09-06T16:30:21.265Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:30:31.254" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:30:21.264" grpc.time_ms=0.258 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1928gitlab # [ 250.051492] gitaly[921]: time="2026-09-06T16:30:21.283Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=941 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-06T16:30:31.262" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:21.272" grpc.time_ms=10.947 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1929gitlab # [ 250.147345] gitaly[921]: time="2026-09-06T16:30:21.379Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=342 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-06T16:30:51.072" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:21.372" grpc.time_ms=6.644 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1930gitlab # [ 250.210437] gitaly[921]: time="2026-09-06T16:30:21.442Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-06T16:30:31.001" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:30:21.442" grpc.time_ms=0.408 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1931gitlab # [ 250.241567] gitaly[921]: time="2026-09-06T16:30:21.473Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=342 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-06T16:30:31.025" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-06T16:30:21.466" grpc.time_ms=7.324 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1932gitlab # [ 250.270443] gitaly[921]: time="2026-09-06T16:30:21.502Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:30:31.051" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:21.492" grpc.time_ms=10.615 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1933gitlab # [ 250.297739] gitaly[921]: time="2026-09-06T16:30:21.530Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:30:31.081" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:21.521" grpc.time_ms=8.672 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1934gitlab # [ 250.325952] gitaly[921]: time="2026-09-06T16:30:21.558Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=956 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-06T16:30:51.244" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:21.544" grpc.time_ms=13.793 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1935gitlab # [ 250.712298] gitaly[921]: time="2026-09-06T16:30:21.944Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-06T16:30:31.002" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:21.943" grpc.time_ms=1.338 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1936gitlab # [ 250.739084] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:21.970Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 12bf65ece35b0516d0c8e2591b41125a05ab1df8 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"7de861b1183ba3bb87acb865","created_at":"2026-09-06T16:29:33.209Z","trace_propagation_headers":{"sentry-trace":"0b62de9e8d014266b4c693d30fa5d97e-a1637ba99f234b54","baggage":"sentry-trace_id=0b62de9e8d014266b4c693d30fa5d97e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRVF92VENKA2GZPJSCYPBZ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:29:33.212Z","job_size_bytes":185,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-7de861b1183ba3bb87acb865: done: 0.987491 sec","job_status":"done","queue_duration_s":47.770055,"scheduling_latency_s":47.770068,"gitaly_calls":13,"gitaly_duration_s":0.330174,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.057785,"redis_read_bytes":2103,"redis_write_bytes":21748,"redis_cache_calls":15,"redis_cache_duration_s":0.005004,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.006705,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001631,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7836,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004682,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.028423,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.01134,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.053,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.191,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.213,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.053,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.573311,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000454806,"exclusive_lock_hold_duration_s":0.079081839,"duration_s":0.987491,"completed_at":"2026-09-06T16:30:21.970Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.094199,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1937gitlab # [ 250.799772] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:21.971Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"dec6210c9f2c10331a6b4058","created_at":"2026-09-06T16:29:37.843Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-52c8f70e7e994583","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:37.849Z","job_size_bytes":65,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-dec6210c9f2c10331a6b4058: start","job_status":"start","queue_duration_s":44.12277,"scheduling_latency_s":44.122785}1938gitlab # [ 250.814380] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:22.045Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"dec6210c9f2c10331a6b4058","created_at":"2026-09-06T16:29:37.843Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-52c8f70e7e994583","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:37.849Z","job_size_bytes":65,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-dec6210c9f2c10331a6b4058: done: 0.074197 sec","job_status":"done","queue_duration_s":44.12277,"scheduling_latency_s":44.122785,"redis_calls":16,"redis_duration_s":0.0038120000000000003,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001691,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001621,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0005,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039128,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000605105,"duration_s":0.074197,"completed_at":"2026-09-06T16:30:22.045Z","load_balancing_strategy":"primary","db_duration_s":0.026389,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1939gitlab # [ 250.816477] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:22.048Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"f780b9665d79535094320927","created_at":"2026-09-06T16:29:37.849Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-52c8f70e7e994583","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:37.852Z","job_size_bytes":65,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-f780b9665d79535094320927: start","job_status":"start","queue_duration_s":44.195969,"scheduling_latency_s":44.196}1940gitlab # [ 250.918925] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:22.150Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"f780b9665d79535094320927","created_at":"2026-09-06T16:29:37.849Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-52c8f70e7e994583","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:37.852Z","job_size_bytes":65,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-f780b9665d79535094320927: done: 0.102157 sec","job_status":"done","queue_duration_s":44.195969,"scheduling_latency_s":44.196,"redis_calls":21,"redis_duration_s":0.00601,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002701,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001871,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001438,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.068135,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000511238,"duration_s":0.102157,"completed_at":"2026-09-06T16:30:22.150Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022072,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1941gitlab # [ 250.921205] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:22.152Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"fca80a33cab0ea431be380ab","created_at":"2026-09-06T16:29:37.953Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-52c8f70e7e994583","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:37.958Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-fca80a33cab0ea431be380ab: start","job_status":"start","queue_duration_s":44.194526,"scheduling_latency_s":44.194551}1942gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0'1943gitlab # [ 251.046333] nixbot[2644]: INFO: ::ffff:127.0.0.1:57076 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 HTTP/1.1" 200 OK1944gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0', in 0.04 seconds)1945gitlab # [ 251.081564] gitaly[921]: time="2026-09-06T16:30:22.313Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:30:22.313" grpc.time_ms=0.326 pid=921 span.kind=server system=grpc1946gitlab # [ 251.135633] gitaly[921]: time="2026-09-06T16:30:22.367Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:30:22.367" grpc.time_ms=0.634 pid=921 span.kind=server system=grpc1947gitlab # [ 251.163061] gitaly[921]: time="2026-09-06T16:30:22.395Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:30:22.395" grpc.time_ms=0.274 pid=921 span.kind=server system=grpc1948gitlab # [ 251.179171] gitaly[921]: time="2026-09-06T16:30:22.411Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 diskcache=affae9fc-cc6b-478c-a1b8-49b36f954110 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:30:52.074" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:22.274" pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1949gitlab # [ 251.179777] gitaly[921]: time="2026-09-06T16:30:22.411Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=160 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7789 command.oublock=0 command.real_time_ms=130 command.spawn_token_fork_ms=0 command.system_time_ms=115 command.user_time_ms=45 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:30:52.074" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:22.274" grpc.time_ms=136.859 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1950gitlab # [ 251.353589] gitaly[921]: time="2026-09-06T16:30:22.585Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:30:52.284" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:22.584" grpc.time_ms=0.953 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1951gitlab # [ 251.384442] gitaly[921]: time="2026-09-06T16:30:22.616Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:30:52.315" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:22.615" grpc.time_ms=0.754 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1952gitlab # [ 251.402778] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:22.635Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"fca80a33cab0ea431be380ab","created_at":"2026-09-06T16:29:37.953Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-52c8f70e7e994583","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:37.958Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2536,"job_status":"rescheduled","message":"PipelineProcessWorker JID-fca80a33cab0ea431be380ab: rescheduled"}1953gitlab # [ 251.409332] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:22.641Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"fca80a33cab0ea431be380ab","created_at":"2026-09-06T16:29:37.953Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-52c8f70e7e994583","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:29:37.958Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-fca80a33cab0ea431be380ab: done: 0.488735 sec","job_status":"done","queue_duration_s":44.194526,"scheduling_latency_s":44.194551,"gitaly_calls":3,"gitaly_duration_s":0.175885,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.025572,"redis_read_bytes":460,"redis_write_bytes":29288,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001514,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001082,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.009634,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.003679,"redis_queues_read_bytes":37,"redis_queues_write_bytes":15575,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.005379,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000665,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000467,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001638,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.056,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.025,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.029,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.025,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.223979,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.444603992,"duration_s":0.488735,"completed_at":"2026-09-06T16:30:22.641Z","load_balancing_strategy":"primary","db_duration_s":0.071772,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1954gitlab # [ 251.423259] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:22.642Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","12bf65ece35b0516d0c8e2591b41125a05ab1df8 d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"78b7ad239a412a45e64bce5f","created_at":"2026-09-06T16:30:20.049Z","trace_propagation_headers":{"sentry-trace":"4f27504bc6814cdd815c0b8bfbca3e58-c05eb32d725a48b1","baggage":"sentry-trace_id=4f27504bc6814cdd815c0b8bfbca3e58,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRWWYX5A86HQ0FEQBYK3DY","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:30:20.051Z","job_size_bytes":185,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-78b7ad239a412a45e64bce5f: start","job_status":"start","queue_duration_s":2.591442,"scheduling_latency_s":2.591456}1955gitlab # [ 251.460994] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:22.650Z","message":"changes: \"12bf65ece35b0516d0c8e2591b41125a05ab1df8 d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 refs/heads/master\\n\""}1956gitlab # [ 251.461688] gitaly[921]: time="2026-09-06T16:30:22.669Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:30:32.222" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:22.663" grpc.time_ms=5.746 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1957gitlab # [ 251.494483] gitaly[921]: time="2026-09-06T16:30:22.726Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:30:32.279" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:30:22.719" grpc.time_ms=6.95 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1958gitlab # [ 251.499674] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:22.731Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"8618bf871b4ba97977caca66","created_at":"2026-09-06T16:30:22.729Z","trace_propagation_headers":{"sentry-trace":"4f27504bc6814cdd815c0b8bfbca3e58-1f6c0b6e3d54487e","baggage":"sentry-trace_id=4f27504bc6814cdd815c0b8bfbca3e58,sentry-release=Unknown"},"correlation_id":"01M1VRWWYX5A86HQ0FEQBYK3DY","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"a32fa439985c8a53e12ea051","job_size_bytes":46,"pid":2536,"job_status":"deduplicated","message":"ProjectCacheWorker JID-8618bf871b4ba97977caca66: deduplicated: until executing","deduplication.type":"until executing"}1959gitlab # [ 251.517098] gitaly[921]: time="2026-09-06T16:30:22.749Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-06T16:30:32.004" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:22.744" grpc.time_ms=4.456 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1960gitlab # [ 251.528069] gitaly[921]: time="2026-09-06T16:30:22.759Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=651 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-06T16:30:52.453" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:22.754" grpc.time_ms=5.24 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1961gitlab # [ 251.539038] gitaly[921]: time="2026-09-06T16:30:22.771Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=344 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-06T16:30:32.025" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-06T16:30:22.765" grpc.time_ms=5.526 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1962gitlab # [ 251.554092] gitaly[921]: time="2026-09-06T16:30:22.786Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:30:32.036" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:22.776" grpc.time_ms=9.786 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1963gitlab # [ 251.626775] gitaly[921]: time="2026-09-06T16:30:22.859Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-06T16:30:32.117" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:22.857" grpc.time_ms=0.931 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1964gitlab # [ 251.641375] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:22.873Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","12bf65ece35b0516d0c8e2591b41125a05ab1df8 d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"78b7ad239a412a45e64bce5f","created_at":"2026-09-06T16:30:20.049Z","trace_propagation_headers":{"sentry-trace":"4f27504bc6814cdd815c0b8bfbca3e58-c05eb32d725a48b1","baggage":"sentry-trace_id=4f27504bc6814cdd815c0b8bfbca3e58,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRWWYX5A86HQ0FEQBYK3DY","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:30:20.051Z","job_size_bytes":185,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-78b7ad239a412a45e64bce5f: done: 0.230607 sec","job_status":"done","queue_duration_s":2.591442,"scheduling_latency_s":2.591456,"gitaly_calls":7,"gitaly_duration_s":0.097486,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.01909,"redis_read_bytes":501,"redis_write_bytes":11943,"redis_cache_calls":9,"redis_cache_duration_s":0.002698,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.000964,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5720,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002521,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.005918,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.006989,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.017,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.017,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.102558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000481905,"exclusive_lock_hold_duration_s":0.059661493,"duration_s":0.230607,"completed_at":"2026-09-06T16:30:22.873Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022211,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1965gitlab # [ 251.645700] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:22.874Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"a32fa439985c8a53e12ea051","created_at":"2026-09-06T16:30:21.099Z","trace_propagation_headers":{"sentry-trace":"0b62de9e8d014266b4c693d30fa5d97e-e23b11d067004ad0","baggage":"sentry-trace_id=0b62de9e8d014266b4c693d30fa5d97e,sentry-release=Unknown"},"correlation_id":"01M1VRVF92VENKA2GZPJSCYPBZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:21.121Z","job_size_bytes":46,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-a32fa439985c8a53e12ea051: start","job_status":"start","queue_duration_s":1.753308,"scheduling_latency_s":1.753328}1966gitlab # [ 251.733948] gitaly[921]: time="2026-09-06T16:30:22.966Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-06T22:30:22.963" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:30:22.963" grpc.time_ms=2.237 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1967gitlab # [ 251.754178] gitaly[921]: time="2026-09-06T16:30:22.986Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=647 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-06T16:30:52.779" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:22.979" grpc.time_ms=6.186 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1968gitlab # [ 251.788039] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:23.019Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"a32fa439985c8a53e12ea051","created_at":"2026-09-06T16:30:21.099Z","trace_propagation_headers":{"sentry-trace":"0b62de9e8d014266b4c693d30fa5d97e-e23b11d067004ad0","baggage":"sentry-trace_id=0b62de9e8d014266b4c693d30fa5d97e,sentry-release=Unknown"},"correlation_id":"01M1VRVF92VENKA2GZPJSCYPBZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:21.121Z","job_size_bytes":46,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-a32fa439985c8a53e12ea051: done: 0.144565 sec","job_status":"done","queue_duration_s":1.753308,"scheduling_latency_s":1.753328,"gitaly_calls":2,"gitaly_duration_s":0.013247,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.005613,"redis_read_bytes":69,"redis_write_bytes":4888,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001005,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000735,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001316,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.002028,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000529,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057656,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000429663,"duration_s":0.144565,"completed_at":"2026-09-06T16:30:23.019Z","load_balancing_strategy":"primary","db_duration_s":0.012259,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1969gitlab # [ 251.832204] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:23.064Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"a62df56fe40e661a45cc6c3e","created_at":"2026-09-06T16:30:21.138Z","trace_propagation_headers":{"sentry-trace":"0b62de9e8d014266b4c693d30fa5d97e-e23b11d067004ad0","baggage":"sentry-trace_id=0b62de9e8d014266b4c693d30fa5d97e,sentry-release=Unknown"},"correlation_id":"01M1VRVF92VENKA2GZPJSCYPBZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:21.143Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-a62df56fe40e661a45cc6c3e: start","job_status":"start","queue_duration_s":1.920319,"scheduling_latency_s":1.920362}1970gitlab # [ 251.888434] gitaly[921]: time="2026-09-06T16:30:23.120Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ diskcache=a01625cc-1892-4de0-ba8d-141cb7154b5d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-06T22:30:23.094" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:23.094" pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1971gitlab # [ 251.897608] gitaly[921]: time="2026-09-06T16:30:23.129Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1286 command.oublock=0 command.real_time_ms=27 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-06T22:30:23.094" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:23.094" grpc.time_ms=26.058 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1972gitlab # [ 251.940695] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:23.172Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"a62df56fe40e661a45cc6c3e","created_at":"2026-09-06T16:30:21.138Z","trace_propagation_headers":{"sentry-trace":"0b62de9e8d014266b4c693d30fa5d97e-e23b11d067004ad0","baggage":"sentry-trace_id=0b62de9e8d014266b4c693d30fa5d97e,sentry-release=Unknown"},"correlation_id":"01M1VRVF92VENKA2GZPJSCYPBZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:21.143Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-a62df56fe40e661a45cc6c3e: done: 0.108549 sec","job_status":"done","queue_duration_s":1.920319,"scheduling_latency_s":1.920362,"gitaly_calls":1,"gitaly_duration_s":0.038096,"redis_calls":15,"redis_duration_s":0.003098,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000393,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001189,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000546,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00097,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.011,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.021,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.07835353399999999,"duration_s":0.108549,"completed_at":"2026-09-06T16:30:23.172Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018416,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1973gitlab # [ 251.942665] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:23.174Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"c7151dda7c44a612e73d5615","created_at":"2026-09-06T16:30:21.259Z","trace_propagation_headers":{"sentry-trace":"0b62de9e8d014266b4c693d30fa5d97e-e23b11d067004ad0","baggage":"sentry-trace_id=0b62de9e8d014266b4c693d30fa5d97e,sentry-release=Unknown"},"correlation_id":"01M1VRVF92VENKA2GZPJSCYPBZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:21.262Z","job_size_bytes":23,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-c7151dda7c44a612e73d5615: start","job_status":"start","queue_duration_s":1.912647,"scheduling_latency_s":1.912671}1974gitlab # [ 252.013889] gitaly[921]: time="2026-09-06T16:30:23.246Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=650 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-06T16:30:53.039" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:23.239" grpc.time_ms=6.149 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1975gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0'1976gitlab # [ 252.025531] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:23.257Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"c7151dda7c44a612e73d5615","created_at":"2026-09-06T16:30:21.259Z","trace_propagation_headers":{"sentry-trace":"0b62de9e8d014266b4c693d30fa5d97e-e23b11d067004ad0","baggage":"sentry-trace_id=0b62de9e8d014266b4c693d30fa5d97e,sentry-release=Unknown"},"correlation_id":"01M1VRVF92VENKA2GZPJSCYPBZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:21.262Z","job_size_bytes":23,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-c7151dda7c44a612e73d5615: done: 0.082662 sec","job_status":"done","queue_duration_s":1.912647,"scheduling_latency_s":1.912671,"gitaly_calls":1,"gitaly_duration_s":0.008949,"redis_calls":16,"redis_duration_s":0.00413,"redis_read_bytes":52,"redis_write_bytes":2047,"redis_queues_calls":1,"redis_queues_duration_s":0.000326,"redis_queues_read_bytes":1,"redis_queues_write_bytes":978,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001141,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.00192,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000743,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.041,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.041,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026499,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.082662,"completed_at":"2026-09-06T16:30:23.257Z","load_balancing_strategy":"primary","db_duration_s":0.041215,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1977gitlab # [ 252.026798] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:23.259Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"61aaae18c2b8f261778c7a98","created_at":"2026-09-06T16:30:21.817Z","trace_propagation_headers":{"sentry-trace":"0b62de9e8d014266b4c693d30fa5d97e-e23b11d067004ad0","baggage":"sentry-trace_id=0b62de9e8d014266b4c693d30fa5d97e,sentry-release=Unknown"},"correlation_id":"01M1VRVF92VENKA2GZPJSCYPBZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6cb46f96214642b21e1cbd3527d4f1c18bb5b7343bbed6ebc73ba70f92fdd37d","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:21.823Z","job_size_bytes":467,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-61aaae18c2b8f261778c7a98: start","job_status":"start","queue_duration_s":1.435384,"scheduling_latency_s":1.435403}1978gitlab # [ 252.047133] gitaly[921]: time="2026-09-06T16:30:23.279Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:30:53.078" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:23.278" grpc.time_ms=0.445 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1979gitlab # [ 252.053457] gitaly[921]: time="2026-09-06T16:30:23.285Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-06T16:30:33.010" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:23.280" grpc.time_ms=4.885 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1980gitlab # [ 252.085829] nixbot[2644]: INFO: ::ffff:127.0.0.1:57084 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 HTTP/1.1" 200 OK1981gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0', in 0.05 seconds)1982gitlab # [ 252.135144] gitaly[921]: time="2026-09-06T16:30:23.366Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:30:53.165" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:23.365" grpc.time_ms=0.855 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1983gitlab # [ 252.154368] gitaly[921]: time="2026-09-06T16:30:23.386Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=115 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-06T16:30:53.182" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:23.383" grpc.time_ms=3.037 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1984gitlab # [ 252.219309] gitaly[921]: time="2026-09-06T16:30:23.451Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:30:33.174" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:23.444" grpc.time_ms=6.7 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1985gitlab # [ 252.657381] gitaly[921]: time="2026-09-06T16:30:23.889Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=961 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-06T16:30:53.670" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:23.870" grpc.time_ms=18.503 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1986gitlab # [ 252.674993] gitaly[921]: time="2026-09-06T16:30:23.907Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVF92VENKA2GZPJSCYPBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-06T16:30:53.691" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:23.892" grpc.time_ms=15.013 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1987gitlab # [ 252.749701] sidekiq-cluster[2536]: {"severity":"WARN","time":"2026-09-06T16:30:23.981Z","class":"CreatePipelineWorker","correlation_id":"01M1VRVF92VENKA2GZPJSCYPBZ","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"12bf65ece35b0516d0c8e2591b41125a05ab1df8","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"12bf65ece35b0516d0c8e2591b41125a05ab1df8","ref":"refs/heads/master"},"retry":0}1988gitlab # [ 252.772330] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.003Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"61aaae18c2b8f261778c7a98","created_at":"2026-09-06T16:30:21.817Z","trace_propagation_headers":{"sentry-trace":"0b62de9e8d014266b4c693d30fa5d97e-e23b11d067004ad0","baggage":"sentry-trace_id=0b62de9e8d014266b4c693d30fa5d97e,sentry-release=Unknown"},"correlation_id":"01M1VRVF92VENKA2GZPJSCYPBZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6cb46f96214642b21e1cbd3527d4f1c18bb5b7343bbed6ebc73ba70f92fdd37d","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:21.823Z","job_size_bytes":467,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-61aaae18c2b8f261778c7a98: done: 0.744769 sec","job_status":"done","queue_duration_s":1.435384,"scheduling_latency_s":1.435403,"gitaly_calls":7,"gitaly_duration_s":0.122126,"redis_calls":51,"redis_duration_s":0.026699999999999998,"redis_read_bytes":169,"redis_write_bytes":4167,"redis_cache_calls":4,"redis_cache_duration_s":0.00095,"redis_cache_read_bytes":10,"redis_cache_write_bytes":342,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.004271,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.017678,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002768,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001033,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.540405,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000312609,"duration_s":0.744769,"completed_at":"2026-09-06T16:30:24.003Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.050667,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1989gitlab # [ 252.842213] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.073Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"5a0c0bfa-836a-48a4-9fbf-3f15f82e8bdf\"}"],"class":"WebHookWorker","jid":"93156ea4a42854a51e43d0c3","created_at":"2026-09-06T16:30:21.919Z","trace_propagation_headers":{"sentry-trace":"0b62de9e8d014266b4c693d30fa5d97e-e23b11d067004ad0","baggage":"sentry-trace_id=0b62de9e8d014266b4c693d30fa5d97e,sentry-release=Unknown"},"correlation_id":"01M1VRVF92VENKA2GZPJSCYPBZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ab5f734c5672b3a54eee6d8d190d4c9f1eecbf8bcdcd37788688c4ad7cdec784","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:21.924Z","job_size_bytes":1751,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-93156ea4a42854a51e43d0c3: start","job_status":"start","queue_duration_s":2.148763,"scheduling_latency_s":2.148813}1990gitlab # [ 253.007315] nixbot[2644]: INFO: ::ffff:127.0.0.1:37044 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1991gitlab # [ 253.029905] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.261Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"5a0c0bfa-836a-48a4-9fbf-3f15f82e8bdf\"}"],"class":"WebHookWorker","jid":"93156ea4a42854a51e43d0c3","created_at":"2026-09-06T16:30:21.919Z","trace_propagation_headers":{"sentry-trace":"0b62de9e8d014266b4c693d30fa5d97e-e23b11d067004ad0","baggage":"sentry-trace_id=0b62de9e8d014266b4c693d30fa5d97e,sentry-release=Unknown"},"correlation_id":"01M1VRVF92VENKA2GZPJSCYPBZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ab5f734c5672b3a54eee6d8d190d4c9f1eecbf8bcdcd37788688c4ad7cdec784","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:21.924Z","job_size_bytes":1751,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-93156ea4a42854a51e43d0c3: done: 0.18877 sec","job_status":"done","queue_duration_s":2.148763,"scheduling_latency_s":2.148813,"redis_calls":19,"redis_duration_s":0.006048,"redis_read_bytes":32,"redis_write_bytes":5367,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001116,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000571,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003016,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001345,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.05472455299999979,"cpu_s":0.110904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000425473,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.18877,"completed_at":"2026-09-06T16:30:24.261Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004342,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1992gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0'1993gitlab # [ 253.108860] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.340Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"f58808d53641acb7d9b28d32","created_at":"2026-09-06T16:30:22.255Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.260Z","job_size_bytes":23,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-f58808d53641acb7d9b28d32: start","job_status":"start","queue_duration_s":2.079684,"scheduling_latency_s":2.079724}1994gitlab # [ 253.132927] gitaly[921]: time="2026-09-06T16:30:24.365Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VRX1GJ996KK8PQBQ1SVZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:30:24.346" grpc.time_ms=18.953 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc1995gitlab # [ 253.135155] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:24 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 921996gitlab # [ 253.149183] nixbot[2644]: INFO: ::ffff:127.0.0.1:37052 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 HTTP/1.1" 200 OK1997gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0', in 0.04 seconds)1998gitlab # [ 253.204719] gitaly[921]: time="2026-09-06T16:30:24.436Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX1KJJGGNCGENYV971J2Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:30:24.429" pid=921 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1999gitlab # [ 253.221589] gitaly[921]: time="2026-09-06T16:30:24.436Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX1KJJGGNCGENYV971J2Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:30:24.429" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc2000gitlab # [ 253.222353] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.441Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"f58808d53641acb7d9b28d32","created_at":"2026-09-06T16:30:22.255Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.260Z","job_size_bytes":23,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-f58808d53641acb7d9b28d32: done: 0.101093 sec","job_status":"done","queue_duration_s":2.079684,"scheduling_latency_s":2.079724,"redis_calls":10,"redis_duration_s":0.020609,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000252,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.019319,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001038,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045709,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000230476,"duration_s":0.101093,"completed_at":"2026-09-06T16:30:24.441Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013132,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2001gitlab # [ 253.262886] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.442Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"fb54724856519e55786e85ae","created_at":"2026-09-06T16:30:22.437Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.441Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-fb54724856519e55786e85ae: start","job_status":"start","queue_duration_s":2.001547,"scheduling_latency_s":2.001562}2002gitlab # [ 253.263808] gitlab-workhorse[826]: time="2026-09-06T16:30:24Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672003gitlab # [ 253.264537] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:24 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 672004gitlab # [ 253.265187] gitaly[921]: time="2026-09-06T16:30:24.437Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX1KJJGGNCGENYV971J2Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:30:24.429" grpc.time_ms=7.221 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc2005gitlab # [ 253.307613] gitaly[921]: time="2026-09-06T16:30:24.539Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX1NP5QR9R9PWZVVZABGJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:30:24.470" pid=921 remote_ip="2001:db8:1::1" request_sha=ade8518baf04ffba596faf9eb7c5eaa5fa2d7af8 response_bytes=337 span.kind=server system=grpc2006gitlab # [ 253.308754] gitaly[921]: time="2026-09-06T16:30:24.540Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1186 command.oublock=0 command.real_time_ms=67 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX1NP5QR9R9PWZVVZABGJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:30:24.470" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc2007gitlab # [ 253.309637] gitlab-workhorse[826]: time="2026-09-06T16:30:24Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3372008gitlab # [ 253.310549] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:24 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.55.0" 1012009gitlab # [ 253.311588] gitaly[921]: time="2026-09-06T16:30:24.540Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1186 command.oublock=0 command.real_time_ms=67 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX1NP5QR9R9PWZVVZABGJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:30:24.470" grpc.time_ms=69.811 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc2010gitlab # [ 253.318168] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.550Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"fb54724856519e55786e85ae","created_at":"2026-09-06T16:30:22.437Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.441Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-fb54724856519e55786e85ae: done: 0.107297 sec","job_status":"done","queue_duration_s":2.001547,"scheduling_latency_s":2.001562,"redis_calls":10,"redis_duration_s":0.00214,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000271,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001228,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000641,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.085,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.085,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029948,"duration_s":0.107297,"completed_at":"2026-09-06T16:30:24.550Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.085005,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2011gitlab # [ 253.372125] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.551Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"d1e4d795eb7af1603c18a9f2","created_at":"2026-09-06T16:30:22.449Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.453Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-d1e4d795eb7af1603c18a9f2: start","job_status":"start","queue_duration_s":2.097703,"scheduling_latency_s":2.097717}2012gitlab # [ 253.372705] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.575Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"d1e4d795eb7af1603c18a9f2","created_at":"2026-09-06T16:30:22.449Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.453Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-d1e4d795eb7af1603c18a9f2: done: 0.024666 sec","job_status":"done","queue_duration_s":2.097703,"scheduling_latency_s":2.097717,"redis_calls":10,"redis_duration_s":0.001339,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000276,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00079,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000273,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000216787,"duration_s":0.024666,"completed_at":"2026-09-06T16:30:24.575Z","load_balancing_strategy":"primary","db_duration_s":0.004863,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2013gitlab # [ 253.412138] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.576Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"0d5d4067dfd79e22607a2bba","created_at":"2026-09-06T16:30:22.461Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.464Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-0d5d4067dfd79e22607a2bba: start","job_status":"start","queue_duration_s":2.112095,"scheduling_latency_s":2.112109}2014gitlab # [ 253.412747] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.600Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"0d5d4067dfd79e22607a2bba","created_at":"2026-09-06T16:30:22.461Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.464Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-0d5d4067dfd79e22607a2bba: done: 0.023722 sec","job_status":"done","queue_duration_s":2.112095,"scheduling_latency_s":2.112109,"redis_calls":13,"redis_duration_s":0.001707,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000272,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000791,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000356,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000288,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00021092,"duration_s":0.023722,"completed_at":"2026-09-06T16:30:24.600Z","load_balancing_strategy":"primary","db_duration_s":0.003299,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2015gitlab # [ 253.461124] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.601Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"4614077dc045ad825a192c44","created_at":"2026-09-06T16:30:22.465Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.467Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-4614077dc045ad825a192c44: start","job_status":"start","queue_duration_s":2.133631,"scheduling_latency_s":2.133643}2016gitlab # [ 253.466572] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.623Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"4614077dc045ad825a192c44","created_at":"2026-09-06T16:30:22.465Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.467Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-4614077dc045ad825a192c44: done: 0.021444 sec","job_status":"done","queue_duration_s":2.133631,"scheduling_latency_s":2.133643,"redis_calls":10,"redis_duration_s":0.0028970000000000003,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000261,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000799,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001837,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013413,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208127,"duration_s":0.021444,"completed_at":"2026-09-06T16:30:24.623Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002732,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2017gitlab # [ 253.509762] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.624Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"93fbbca3d4b005df174a2047","created_at":"2026-09-06T16:30:22.468Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.471Z","job_size_bytes":19,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-93fbbca3d4b005df174a2047: start","job_status":"start","queue_duration_s":2.152415,"scheduling_latency_s":2.152428}2018gitlab # [ 253.557751] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.642Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"93fbbca3d4b005df174a2047","created_at":"2026-09-06T16:30:22.468Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.471Z","job_size_bytes":19,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-93fbbca3d4b005df174a2047: done: 0.018371 sec","job_status":"done","queue_duration_s":2.152415,"scheduling_latency_s":2.152428,"redis_calls":11,"redis_duration_s":0.001415,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00027,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000889,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000256,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011249,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000212876,"duration_s":0.018371,"completed_at":"2026-09-06T16:30:24.642Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002996,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2019gitlab # [ 253.561143] nixbot[2644]: {"ts": "2026-09-06T16:30:24.719Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}2020gitlab # [ 253.562400] gitaly[921]: time="2026-09-06T16:30:24.778Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX1YSRAY6RR931KF6TF3H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:30:54.777" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:24.777" grpc.time_ms=0.596 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2021gitlab # [ 253.606301] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.643Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"a2fe998e3738c6ca8ac035e0","created_at":"2026-09-06T16:30:22.473Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.476Z","job_size_bytes":66,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-a2fe998e3738c6ca8ac035e0: start","job_status":"start","queue_duration_s":2.16673,"scheduling_latency_s":2.166743}2022gitlab # [ 253.606918] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.724Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"a2fe998e3738c6ca8ac035e0","created_at":"2026-09-06T16:30:22.473Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.476Z","job_size_bytes":66,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-a2fe998e3738c6ca8ac035e0: done: 0.080923 sec","job_status":"done","queue_duration_s":2.16673,"scheduling_latency_s":2.166743,"redis_calls":10,"redis_duration_s":0.004144,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000783,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002853,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000508,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.052,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.049,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000734171,"duration_s":0.080923,"completed_at":"2026-09-06T16:30:24.724Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.052509,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2023gitlab # [ 253.646350] gitaly[921]: time="2026-09-06T16:30:24.819Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRX1YSRAY6RR931KF6TF3H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:30:34.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:24.813" grpc.time_ms=5.93 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2024gitlab # [ 253.646944] nixbot[2644]: {"ts": "2026-09-06T16:30:24.862Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/12bf65ece35b0516d0c8e2591b41125a05ab1df8 \"HTTP/1.1 201 Created\""}2025gitlab # [ 253.647773] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:24 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/12bf65ece35b0516d0c8e2591b41125a05ab1df8 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 1322026gitlab # [ 253.648396] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.725Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"1d55817dbffc55b6f035a140","created_at":"2026-09-06T16:30:22.477Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.480Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-1d55817dbffc55b6f035a140: start","job_status":"start","queue_duration_s":2.244867,"scheduling_latency_s":2.244885}2027gitlab # [ 253.692845] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.757Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"1d55817dbffc55b6f035a140","created_at":"2026-09-06T16:30:22.477Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.480Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-1d55817dbffc55b6f035a140: done: 0.032445 sec","job_status":"done","queue_duration_s":2.244867,"scheduling_latency_s":2.244885,"redis_calls":10,"redis_duration_s":0.0024319999999999997,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000278,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001865,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000289,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000275175,"duration_s":0.032445,"completed_at":"2026-09-06T16:30:24.757Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005336,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2028gitlab # [ 253.698826] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.760Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"9d03a983238c3ff6d4f6430f","created_at":"2026-09-06T16:30:22.487Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.491Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-9d03a983238c3ff6d4f6430f: start","job_status":"start","queue_duration_s":2.268922,"scheduling_latency_s":2.268935}2029gitlab # [ 253.738322] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.790Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"9d03a983238c3ff6d4f6430f","created_at":"2026-09-06T16:30:22.487Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.491Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-9d03a983238c3ff6d4f6430f: done: 0.030181 sec","job_status":"done","queue_duration_s":2.268922,"scheduling_latency_s":2.268935,"redis_calls":10,"redis_duration_s":0.004306,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000399,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002472,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001435,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01465,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000917714,"duration_s":0.030181,"completed_at":"2026-09-06T16:30:24.790Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008547,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2030gitlab # [ 253.782287] gitaly[921]: time="2026-09-06T16:30:24.906Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX231HMXVX6S5RKVE5XZ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:30:54.005" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:24.905" grpc.time_ms=0.612 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2031gitlab # [ 253.782905] nixbot[2644]: {"ts": "2026-09-06T16:30:24.973Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/12bf65ece35b0516d0c8e2591b41125a05ab1df8 \"HTTP/1.1 201 Created\""}2032gitlab # [ 253.783435] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.791Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"cb459f39fea81d00e53c9e9a","created_at":"2026-09-06T16:30:22.492Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.496Z","job_size_bytes":6,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-cb459f39fea81d00e53c9e9a: start","job_status":"start","queue_duration_s":2.29508,"scheduling_latency_s":2.295092}2033gitlab # [ 253.784066] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.812Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"cb459f39fea81d00e53c9e9a","created_at":"2026-09-06T16:30:22.492Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.496Z","job_size_bytes":6,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-cb459f39fea81d00e53c9e9a: done: 0.020753 sec","job_status":"done","queue_duration_s":2.29508,"scheduling_latency_s":2.295092,"redis_calls":10,"redis_duration_s":0.005746,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00229,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002497,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000959,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008673,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000553701,"duration_s":0.020753,"completed_at":"2026-09-06T16:30:24.812Z","load_balancing_strategy":"primary","db_duration_s":0.003575,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2034gitlab # [ 253.824859] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:24 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/12bf65ece35b0516d0c8e2591b41125a05ab1df8 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1072035gitlab # [ 253.825681] gitaly[921]: time="2026-09-06T16:30:24.938Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1VRX231HMXVX6S5RKVE5XZ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:30:34.032" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:24.933" grpc.time_ms=5.43 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2036gitlab # [ 253.865572] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.814Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"1d0f87302a18502f08d7945d","created_at":"2026-09-06T16:30:22.497Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.501Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-1d0f87302a18502f08d7945d: start","job_status":"start","queue_duration_s":2.312553,"scheduling_latency_s":2.312566}2037gitlab # [ 253.866215] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.842Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"1d0f87302a18502f08d7945d","created_at":"2026-09-06T16:30:22.497Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.501Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-1d0f87302a18502f08d7945d: done: 0.028158 sec","job_status":"done","queue_duration_s":2.312553,"scheduling_latency_s":2.312566,"redis_calls":10,"redis_duration_s":0.002014,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00028,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001244,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00049,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017654,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00022852,"duration_s":0.028158,"completed_at":"2026-09-06T16:30:24.842Z","load_balancing_strategy":"primary","db_duration_s":0.005525,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2038gitlab # [ 253.909426] nixbot[2644]: {"ts": "2026-09-06T16:30:25.074Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/12bf65ece35b0516d0c8e2591b41125a05ab1df8 \"HTTP/1.1 201 Created\""}2039gitlab # [ 253.910079] gitaly[921]: time="2026-09-06T16:30:25.012Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX26HASE30FZP5Z0S520F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:30:55.011" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:25.011" grpc.time_ms=0.56 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2040gitlab # [ 253.910702] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:25 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/12bf65ece35b0516d0c8e2591b41125a05ab1df8 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 952041gitlab # [ 253.911160] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:25 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 372042gitlab # [ 253.911808] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.843Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"03d25f711956b7504bbf9da3","created_at":"2026-09-06T16:30:22.572Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.576Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-03d25f711956b7504bbf9da3: start","job_status":"start","queue_duration_s":2.266981,"scheduling_latency_s":2.266996}2043gitlab # [ 253.916587] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.893Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"03d25f711956b7504bbf9da3","created_at":"2026-09-06T16:30:22.572Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.576Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-03d25f711956b7504bbf9da3: done: 0.049517 sec","job_status":"done","queue_duration_s":2.266981,"scheduling_latency_s":2.266996,"redis_calls":14,"redis_duration_s":0.002773,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000261,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001578,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031881,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000689194,"duration_s":0.049517,"completed_at":"2026-09-06T16:30:24.893Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009017,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2044gitlab # [ 253.964326] gitaly[921]: time="2026-09-06T16:30:25.040Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VRX26HASE30FZP5Z0S520F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:30:35.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:25.034" grpc.time_ms=6.325 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2045gitlab # [ 253.967295] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:24.894Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"09039734b8c31b4e6bd0b287","created_at":"2026-09-06T16:30:22.635Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.636Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-09039734b8c31b4e6bd0b287: start","job_status":"start","queue_duration_s":2.257655,"scheduling_latency_s":2.257669}2046gitlab # [ 253.981574] gitaly[921]: time="2026-09-06T16:30:25.133Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VRX2A8F9M60DF7AEHSHN7K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:30:25.127" grpc.time_ms=5.472 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc2047gitlab # [ 254.039515] gitaly[921]: time="2026-09-06T16:30:25.271Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:30:25.271" grpc.time_ms=0.413 pid=921 span.kind=server system=grpc2048gitlab # [ 254.050864] gitaly[921]: time="2026-09-06T16:30:25.283Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX2BFB59J4PKWSHMKE6P5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:30:25.267" pid=921 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2049gitlab # [ 254.051546] gitaly[921]: time="2026-09-06T16:30:25.283Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=10 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX2BFB59J4PKWSHMKE6P5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:30:25.267" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc2050gitlab # [ 254.052333] gitaly[921]: time="2026-09-06T16:30:25.283Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=10 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX2BFB59J4PKWSHMKE6P5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:30:25.267" grpc.time_ms=15.554 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc2051gitlab # [ 254.084358] gitlab-workhorse[826]: time="2026-09-06T16:30:25Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672052gitlab # [ 254.085069] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:25 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1812053gitlab # [ 254.093678] gitaly[921]: time="2026-09-06T16:30:25.326Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:30:25.325" grpc.time_ms=0.357 pid=921 span.kind=server system=grpc2054gitlab # [ 254.122461] gitaly[921]: time="2026-09-06T16:30:25.354Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:30:25.354" grpc.time_ms=0.428 pid=921 span.kind=server system=grpc2055gitlab # [ 254.138479] gitaly[921]: time="2026-09-06T16:30:25.370Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 diskcache=278a877a-4704-4f15-96f6-289b051bda19 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:30:54.750" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:24.950" pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2056gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0'2057gitlab # [ 254.138998] gitaly[921]: time="2026-09-06T16:30:25.371Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=103 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=819200 command.minflt=7309 command.oublock=0 command.real_time_ms=413 command.spawn_token_fork_ms=0 command.system_time_ms=78 command.user_time_ms=25 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:30:54.750" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:24.950" grpc.time_ms=420.322 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2058gitlab # [ 254.184469] gitaly[921]: time="2026-09-06T16:30:25.416Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:30:55.215" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:25.415" grpc.time_ms=0.755 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2059gitlab # [ 254.200026] nixbot[2644]: INFO: ::ffff:127.0.0.1:37060 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 HTTP/1.1" 200 OK2060gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0', in 0.05 seconds)2061gitlab # [ 254.208199] gitaly[921]: time="2026-09-06T16:30:25.440Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:30:55.239" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:25.439" grpc.time_ms=0.507 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2062gitlab # [ 254.215764] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:25.447Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"29f30e8d0282c235a09da2f9","created_at":"2026-09-06T16:30:25.446Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-dd664bbd722b405d","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"09039734b8c31b4e6bd0b287","job_size_bytes":3,"pid":2536,"job_status":"deduplicated","message":"PipelineProcessWorker JID-29f30e8d0282c235a09da2f9: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2063gitlab # [ 254.216834] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:25.449Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"09039734b8c31b4e6bd0b287","created_at":"2026-09-06T16:30:22.635Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.636Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2536,"job_status":"rescheduled","message":"PipelineProcessWorker JID-09039734b8c31b4e6bd0b287: rescheduled"}2064gitlab # [ 254.221951] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:25.454Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"09039734b8c31b4e6bd0b287","created_at":"2026-09-06T16:30:22.635Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-6651fbb93b3842ca","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.636Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-09039734b8c31b4e6bd0b287: done: 0.559856 sec","job_status":"done","queue_duration_s":2.257655,"scheduling_latency_s":2.257669,"gitaly_calls":3,"gitaly_duration_s":0.427964,"redis_calls":69,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.011961,"redis_read_bytes":231,"redis_write_bytes":18468,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.001215,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000707,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.00237,"redis_queues_read_bytes":29,"redis_queues_write_bytes":12314,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.003881,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000251,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000963,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001359,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.008,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.008,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.099526,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.529145185,"duration_s":0.559856,"completed_at":"2026-09-06T16:30:25.454Z","load_balancing_strategy":"primary","db_duration_s":0.027558,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2065gitlab # [ 254.262735] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:25.455Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"3ca32de4d4e0a0870125075e","created_at":"2026-09-06T16:30:22.738Z","trace_propagation_headers":{"sentry-trace":"4f27504bc6814cdd815c0b8bfbca3e58-1f6c0b6e3d54487e","baggage":"sentry-trace_id=4f27504bc6814cdd815c0b8bfbca3e58,sentry-release=Unknown"},"correlation_id":"01M1VRWWYX5A86HQ0FEQBYK3DY","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"a62df56fe40e661a45cc6c3e","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.740Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-3ca32de4d4e0a0870125075e: start","job_status":"start","queue_duration_s":2.714476,"scheduling_latency_s":2.71449}2066gitlab # [ 254.317618] gitaly[921]: time="2026-09-06T16:30:25.549Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY diskcache=0b8d6cbf-e11e-4cb8-839a-1a6c550901be grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-06T22:30:25.536" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:25.537" pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2067gitlab # [ 254.318113] gitaly[921]: time="2026-09-06T16:30:25.550Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=316 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-06T22:30:25.536" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:25.537" grpc.time_ms=12.896 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2068gitlab # [ 254.340340] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:25.572Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"3ca32de4d4e0a0870125075e","created_at":"2026-09-06T16:30:22.738Z","trace_propagation_headers":{"sentry-trace":"4f27504bc6814cdd815c0b8bfbca3e58-1f6c0b6e3d54487e","baggage":"sentry-trace_id=4f27504bc6814cdd815c0b8bfbca3e58,sentry-release=Unknown"},"correlation_id":"01M1VRWWYX5A86HQ0FEQBYK3DY","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"a62df56fe40e661a45cc6c3e","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.740Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-3ca32de4d4e0a0870125075e: done: 0.116997 sec","job_status":"done","queue_duration_s":2.714476,"scheduling_latency_s":2.71449,"gitaly_calls":1,"gitaly_duration_s":0.084473,"redis_calls":12,"redis_duration_s":0.002106,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00036,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000838,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02583,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.100200724,"duration_s":0.116997,"completed_at":"2026-09-06T16:30:25.572Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004091,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2069gitlab # [ 254.381851] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:25.613Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"5c207c8cc18bfe3235bd3218","created_at":"2026-09-06T16:30:22.825Z","trace_propagation_headers":{"sentry-trace":"4f27504bc6814cdd815c0b8bfbca3e58-1f6c0b6e3d54487e","baggage":"sentry-trace_id=4f27504bc6814cdd815c0b8bfbca3e58,sentry-release=Unknown"},"correlation_id":"01M1VRWWYX5A86HQ0FEQBYK3DY","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:a3af20b44716c421fed470bcdde9c832cef354b8ddba559bd45b3f25b7e0e483","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.828Z","job_size_bytes":467,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-5c207c8cc18bfe3235bd3218: start","job_status":"start","queue_duration_s":2.785453,"scheduling_latency_s":2.785489}2070gitlab # [ 254.432474] gitaly[921]: time="2026-09-06T16:30:25.664Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:30:55.462" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:25.663" grpc.time_ms=0.91 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2071gitlab # [ 254.458247] gitaly[921]: time="2026-09-06T16:30:25.690Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-06T16:30:35.008" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:25.679" grpc.time_ms=11.293 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2072gitlab # [ 254.510106] gitaly[921]: time="2026-09-06T16:30:25.742Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:30:55.540" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:25.740" grpc.time_ms=1.388 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2073gitlab # [ 254.518307] gitaly[921]: time="2026-09-06T16:30:25.750Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=37 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-06T16:30:55.546" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:25.747" grpc.time_ms=3.188 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2074gitlab # [ 254.555688] gitaly[921]: time="2026-09-06T16:30:25.788Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:30:35.036" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:25.777" grpc.time_ms=10.648 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2075gitlab # [ 254.955887] gitaly[921]: time="2026-09-06T16:30:26.188Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=967 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-06T16:30:56.068" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:26.168" grpc.time_ms=19.27 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2076gitlab # [ 254.969935] gitaly[921]: time="2026-09-06T16:30:26.202Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VRWWYX5A86HQ0FEQBYK3DY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-06T16:30:56.090" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:26.190" grpc.time_ms=11.602 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2077gitlab # [ 255.031378] sidekiq-cluster[2536]: {"severity":"WARN","time":"2026-09-06T16:30:26.262Z","class":"CreatePipelineWorker","correlation_id":"01M1VRWWYX5A86HQ0FEQBYK3DY","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"12bf65ece35b0516d0c8e2591b41125a05ab1df8","after":"d5b04fed53c5e317aeaca026c0c710c04f7ba3b0","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"d5b04fed53c5e317aeaca026c0c710c04f7ba3b0","ref":"refs/heads/master"},"retry":0}2078gitlab # [ 255.038503] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.270Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"5c207c8cc18bfe3235bd3218","created_at":"2026-09-06T16:30:22.825Z","trace_propagation_headers":{"sentry-trace":"4f27504bc6814cdd815c0b8bfbca3e58-1f6c0b6e3d54487e","baggage":"sentry-trace_id=4f27504bc6814cdd815c0b8bfbca3e58,sentry-release=Unknown"},"correlation_id":"01M1VRWWYX5A86HQ0FEQBYK3DY","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:a3af20b44716c421fed470bcdde9c832cef354b8ddba559bd45b3f25b7e0e483","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.828Z","job_size_bytes":467,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-5c207c8cc18bfe3235bd3218: done: 0.656332 sec","job_status":"done","queue_duration_s":2.785453,"scheduling_latency_s":2.785489,"gitaly_calls":7,"gitaly_duration_s":0.133067,"redis_calls":33,"redis_duration_s":0.00666,"redis_read_bytes":185,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000805,"redis_cache_read_bytes":44,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001568,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003073,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001214,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.48173,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.656332,"completed_at":"2026-09-06T16:30:26.270Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022386,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2079gitlab # [ 255.090000] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.272Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"0a7be13c-3157-4095-aaea-ffbf7a12d71d\"}"],"class":"WebHookWorker","jid":"90ec96453585b61834523f2d","created_at":"2026-09-06T16:30:22.845Z","trace_propagation_headers":{"sentry-trace":"4f27504bc6814cdd815c0b8bfbca3e58-1f6c0b6e3d54487e","baggage":"sentry-trace_id=4f27504bc6814cdd815c0b8bfbca3e58,sentry-release=Unknown"},"correlation_id":"01M1VRWWYX5A86HQ0FEQBYK3DY","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f825194823ce268b0b2116dc209a344344eeef8eb4b84d117e8dbd6779dba2e","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.847Z","job_size_bytes":1749,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-90ec96453585b61834523f2d: start","job_status":"start","queue_duration_s":3.424755,"scheduling_latency_s":3.424788}2080gitlab # [ 255.090728] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.318Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"0a7be13c-3157-4095-aaea-ffbf7a12d71d\"}"],"class":"WebHookWorker","jid":"90ec96453585b61834523f2d","created_at":"2026-09-06T16:30:22.845Z","trace_propagation_headers":{"sentry-trace":"4f27504bc6814cdd815c0b8bfbca3e58-1f6c0b6e3d54487e","baggage":"sentry-trace_id=4f27504bc6814cdd815c0b8bfbca3e58,sentry-release=Unknown"},"correlation_id":"01M1VRWWYX5A86HQ0FEQBYK3DY","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f825194823ce268b0b2116dc209a344344eeef8eb4b84d117e8dbd6779dba2e","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:22.847Z","job_size_bytes":1749,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-90ec96453585b61834523f2d: done: 0.045943 sec","job_status":"done","queue_duration_s":3.424755,"scheduling_latency_s":3.424788,"redis_calls":14,"redis_duration_s":0.003978,"redis_read_bytes":26,"redis_write_bytes":4683,"redis_queues_calls":2,"redis_queues_duration_s":0.000363,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3727,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002143,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001472,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.00858908100002509,"cpu_s":0.026907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.045943,"completed_at":"2026-09-06T16:30:26.318Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002356,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2081gitlab # [ 255.113134] nixbot[2644]: INFO: ::ffff:127.0.0.1:37062 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2082gitlab # [ 255.114234] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.319Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"48c8edf1e6c74d4cba048d3a","created_at":"2026-09-06T16:30:22.998Z","trace_propagation_headers":{"sentry-trace":"0b62de9e8d014266b4c693d30fa5d97e-6d968dd735ef4238","baggage":"sentry-trace_id=0b62de9e8d014266b4c693d30fa5d97e,sentry-release=Unknown"},"correlation_id":"01M1VRVF92VENKA2GZPJSCYPBZ","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:23.004Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-48c8edf1e6c74d4cba048d3a: start","job_status":"start","queue_duration_s":3.315401,"scheduling_latency_s":3.31542}2083gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0'2084gitlab # [ 255.161898] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.361Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"48c8edf1e6c74d4cba048d3a","created_at":"2026-09-06T16:30:22.998Z","trace_propagation_headers":{"sentry-trace":"0b62de9e8d014266b4c693d30fa5d97e-6d968dd735ef4238","baggage":"sentry-trace_id=0b62de9e8d014266b4c693d30fa5d97e,sentry-release=Unknown"},"correlation_id":"01M1VRVF92VENKA2GZPJSCYPBZ","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:23.004Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-48c8edf1e6c74d4cba048d3a: done: 0.041463 sec","job_status":"done","queue_duration_s":3.315401,"scheduling_latency_s":3.31542,"redis_calls":19,"redis_duration_s":0.009949,"redis_read_bytes":22,"redis_write_bytes":4290,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002174,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000618,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2261,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.005274,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001883,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022622,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.008318095999999999,"duration_s":0.041463,"completed_at":"2026-09-06T16:30:26.361Z","load_balancing_strategy":"primary","db_duration_s":0.006449,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2085gitlab # [ 255.168736] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.362Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","93156ea4a42854a51e43d0c3"],"class":"WebHooks::LogExecutionWorker","jid":"3b79399883e963e6217d90e4","created_at":"2026-09-06T16:30:24.242Z","trace_propagation_headers":{"sentry-trace":"0b62de9e8d014266b4c693d30fa5d97e-c884de85f2454032","baggage":"sentry-trace_id=0b62de9e8d014266b4c693d30fa5d97e,sentry-release=Unknown"},"correlation_id":"01M1VRVF92VENKA2GZPJSCYPBZ","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f5e04609aa01a3170919a727a6c10e168fe6e420b2ad8f62a191e7c968ca1161","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:24.249Z","job_size_bytes":2592,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-3b79399883e963e6217d90e4: start","job_status":"start","queue_duration_s":2.112663,"scheduling_latency_s":2.112676}2086gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0', in 0.03 seconds)2087gitlab # [ 255.222596] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.430Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","93156ea4a42854a51e43d0c3"],"class":"WebHooks::LogExecutionWorker","jid":"3b79399883e963e6217d90e4","created_at":"2026-09-06T16:30:24.242Z","trace_propagation_headers":{"sentry-trace":"0b62de9e8d014266b4c693d30fa5d97e-c884de85f2454032","baggage":"sentry-trace_id=0b62de9e8d014266b4c693d30fa5d97e,sentry-release=Unknown"},"correlation_id":"01M1VRVF92VENKA2GZPJSCYPBZ","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f5e04609aa01a3170919a727a6c10e168fe6e420b2ad8f62a191e7c968ca1161","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:24.249Z","job_size_bytes":2592,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-3b79399883e963e6217d90e4: done: 0.067914 sec","job_status":"done","queue_duration_s":2.112663,"scheduling_latency_s":2.112676,"redis_calls":10,"redis_duration_s":0.029318,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000579,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.027829,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00091,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028302,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000286349,"duration_s":0.067914,"completed_at":"2026-09-06T16:30:26.430Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00732,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2088gitlab # [ 255.223688] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.431Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c8c57becd8ea7f9681d8cb18","created_at":"2026-09-06T16:30:24.941Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-dd664bbd722b405d","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:24.943Z","job_size_bytes":23,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c8c57becd8ea7f9681d8cb18: start","job_status":"start","queue_duration_s":1.487319,"scheduling_latency_s":1.487332}2089gitlab # [ 255.271798] nixbot[2644]: INFO: ::ffff:127.0.0.1:37070 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 HTTP/1.1" 200 OK2090gitlab # [ 255.272852] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:26 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1312091gitlab # [ 255.273769] gitaly[921]: time="2026-09-06T16:30:26.474Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRX3H7FSMRNNGR0JV63R33 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:30:26.469" grpc.time_ms=5.095 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc2092gitlab # [ 255.291367] gitaly[921]: time="2026-09-06T16:30:26.523Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX3NCW9WNR91QVTHGYZYC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:30:26.517" pid=921 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2093gitlab # [ 255.291987] gitaly[921]: time="2026-09-06T16:30:26.523Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX3NCW9WNR91QVTHGYZYC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:30:26.517" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc2094gitlab # [ 255.292628] gitaly[921]: time="2026-09-06T16:30:26.524Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX3NCW9WNR91QVTHGYZYC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:30:26.517" grpc.time_ms=7.034 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc2095gitlab # [ 255.293402] gitlab-workhorse[826]: time="2026-09-06T16:30:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672096gitlab # [ 255.293820] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:26 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 472097gitlab # [ 255.336178] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.568Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c8c57becd8ea7f9681d8cb18","created_at":"2026-09-06T16:30:24.941Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-dd664bbd722b405d","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:24.943Z","job_size_bytes":23,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c8c57becd8ea7f9681d8cb18: done: 0.136931 sec","job_status":"done","queue_duration_s":1.487319,"scheduling_latency_s":1.487332,"redis_calls":7,"redis_duration_s":0.099341,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.098944,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000397,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022522,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.136931,"completed_at":"2026-09-06T16:30:26.568Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012924,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2098gitlab # [ 255.337870] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.569Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"7db911e6166333119f8e2bab","created_at":"2026-09-06T16:30:25.375Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-dd664bbd722b405d","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:25.378Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-7db911e6166333119f8e2bab: start","job_status":"start","queue_duration_s":1.191279,"scheduling_latency_s":1.191301}2099gitlab # [ 255.377402] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.589Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"7db911e6166333119f8e2bab","created_at":"2026-09-06T16:30:25.375Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-dd664bbd722b405d","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:25.378Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-7db911e6166333119f8e2bab: done: 0.019515 sec","job_status":"done","queue_duration_s":1.191279,"scheduling_latency_s":1.191301,"redis_calls":7,"redis_duration_s":0.0011849999999999999,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000277,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013211,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019515,"completed_at":"2026-09-06T16:30:26.589Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002902,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2100gitlab # [ 255.387653] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.590Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"a52847ee8a4a9077571f5bc5","created_at":"2026-09-06T16:30:25.383Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-dd664bbd722b405d","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:25.385Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-a52847ee8a4a9077571f5bc5: start","job_status":"start","queue_duration_s":1.204958,"scheduling_latency_s":1.204972}2101gitlab # [ 255.428980] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.606Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"a52847ee8a4a9077571f5bc5","created_at":"2026-09-06T16:30:25.383Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-dd664bbd722b405d","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:25.385Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-a52847ee8a4a9077571f5bc5: done: 0.016371 sec","job_status":"done","queue_duration_s":1.204958,"scheduling_latency_s":1.204972,"redis_calls":7,"redis_duration_s":0.0009069999999999999,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000667,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00024,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011429,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016371,"completed_at":"2026-09-06T16:30:26.606Z","load_balancing_strategy":"primary","db_duration_s":0.002608,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2102gitlab # [ 255.438601] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.607Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"8870f7ff0cad349ad3ef6db9","created_at":"2026-09-06T16:30:25.391Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-dd664bbd722b405d","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:25.392Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-8870f7ff0cad349ad3ef6db9: start","job_status":"start","queue_duration_s":1.215697,"scheduling_latency_s":1.215711}2103gitlab # [ 255.452782] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.627Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"8870f7ff0cad349ad3ef6db9","created_at":"2026-09-06T16:30:25.391Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-dd664bbd722b405d","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:25.392Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-8870f7ff0cad349ad3ef6db9: done: 0.019321 sec","job_status":"done","queue_duration_s":1.215697,"scheduling_latency_s":1.215711,"redis_calls":10,"redis_duration_s":0.001428,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000739,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000375,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000314,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0128,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019321,"completed_at":"2026-09-06T16:30:26.627Z","load_balancing_strategy":"primary","db_duration_s":0.003637,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2104gitlab # [ 255.492296] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.628Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"b6c9f1d8a10b40e27222aa49","created_at":"2026-09-06T16:30:25.393Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-dd664bbd722b405d","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:25.395Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-b6c9f1d8a10b40e27222aa49: start","job_status":"start","queue_duration_s":1.23205,"scheduling_latency_s":1.232063}2105gitlab # [ 255.536128] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.643Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"b6c9f1d8a10b40e27222aa49","created_at":"2026-09-06T16:30:25.393Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-dd664bbd722b405d","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:25.395Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-b6c9f1d8a10b40e27222aa49: done: 0.015354 sec","job_status":"done","queue_duration_s":1.23205,"scheduling_latency_s":1.232063,"redis_calls":7,"redis_duration_s":0.000885,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000644,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000241,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010606,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015354,"completed_at":"2026-09-06T16:30:26.643Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002336,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2106gitlab # [ 255.543250] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.644Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"98cce9c529920216930f874c","created_at":"2026-09-06T16:30:25.396Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-dd664bbd722b405d","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:25.399Z","job_size_bytes":19,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-98cce9c529920216930f874c: start","job_status":"start","queue_duration_s":1.245173,"scheduling_latency_s":1.245187}2107gitlab # [ 255.588247] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.659Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"98cce9c529920216930f874c","created_at":"2026-09-06T16:30:25.396Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-dd664bbd722b405d","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:25.399Z","job_size_bytes":19,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-98cce9c529920216930f874c: done: 0.014806 sec","job_status":"done","queue_duration_s":1.245173,"scheduling_latency_s":1.245187,"redis_calls":8,"redis_duration_s":0.001219,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000982,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000237,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009412,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014806,"completed_at":"2026-09-06T16:30:26.659Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00241,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2108gitlab # [ 255.595297] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.660Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"411e2e775d7b93d8dbd0ae08","created_at":"2026-09-06T16:30:25.400Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-dd664bbd722b405d","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:25.401Z","job_size_bytes":66,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-411e2e775d7b93d8dbd0ae08: start","job_status":"start","queue_duration_s":1.25825,"scheduling_latency_s":1.258263}2109gitlab # [ 255.645397] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.673Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"411e2e775d7b93d8dbd0ae08","created_at":"2026-09-06T16:30:25.400Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-dd664bbd722b405d","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:25.401Z","job_size_bytes":66,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-411e2e775d7b93d8dbd0ae08: done: 0.013921 sec","job_status":"done","queue_duration_s":1.25825,"scheduling_latency_s":1.258263,"redis_calls":7,"redis_duration_s":0.0010040000000000001,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000689,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000315,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009362,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013921,"completed_at":"2026-09-06T16:30:26.673Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00163,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2110gitlab # [ 255.650558] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.675Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"a3bc326e4110068400a45314","created_at":"2026-09-06T16:30:25.402Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-dd664bbd722b405d","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:25.403Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-a3bc326e4110068400a45314: start","job_status":"start","queue_duration_s":1.272651,"scheduling_latency_s":1.272689}2111gitlab # [ 255.699077] gitaly[921]: time="2026-09-06T16:30:26.829Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:30:26.829" grpc.time_ms=0.314 pid=921 span.kind=server system=grpc2112gitlab # [ 255.699716] gitaly[921]: time="2026-09-06T16:30:26.846Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:30:26.845" grpc.time_ms=0.412 pid=921 span.kind=server system=grpc2113gitlab # [ 255.700576] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.740Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"a3bc326e4110068400a45314","created_at":"2026-09-06T16:30:25.402Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-dd664bbd722b405d","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:25.403Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-a3bc326e4110068400a45314: done: 0.065103 sec","job_status":"done","queue_duration_s":1.272651,"scheduling_latency_s":1.272689,"redis_calls":7,"redis_duration_s":0.001408,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001117,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000291,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.046,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.042,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.065103,"completed_at":"2026-09-06T16:30:26.740Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.045945,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2114gitlab # [ 255.741048] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.741Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"e8021be99195f0dbe97624e1","created_at":"2026-09-06T16:30:25.403Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-dd664bbd722b405d","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:25.404Z","job_size_bytes":6,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-e8021be99195f0dbe97624e1: start","job_status":"start","queue_duration_s":1.337599,"scheduling_latency_s":1.337614}2115gitlab # [ 255.741736] gitaly[921]: time="2026-09-06T16:30:26.865Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:30:26.864" grpc.time_ms=0.399 pid=921 span.kind=server system=grpc2116gitlab # [ 255.743032] gitaly[921]: time="2026-09-06T16:30:26.874Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 diskcache=a6134060-eae3-4735-a1a4-01ecb6d9430e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:30:56.709" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:26.809" pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2117gitlab # [ 255.751866] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.751Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"e8021be99195f0dbe97624e1","created_at":"2026-09-06T16:30:25.403Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-dd664bbd722b405d","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:25.404Z","job_size_bytes":6,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-e8021be99195f0dbe97624e1: done: 0.009634 sec","job_status":"done","queue_duration_s":1.337599,"scheduling_latency_s":1.337614,"redis_calls":7,"redis_duration_s":0.0010299999999999999,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000788,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000242,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009634,"completed_at":"2026-09-06T16:30:26.751Z","load_balancing_strategy":"primary","db_duration_s":0.000321,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2118gitlab # [ 255.815158] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.752Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"f590a2366d12573f473dabd3","created_at":"2026-09-06T16:30:25.404Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-dd664bbd722b405d","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:25.405Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-f590a2366d12573f473dabd3: start","job_status":"start","queue_duration_s":1.34691,"scheduling_latency_s":1.346923}2119gitlab # [ 255.815929] gitaly[921]: time="2026-09-06T16:30:26.874Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=64 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7206 command.oublock=0 command.real_time_ms=57 command.spawn_token_fork_ms=0 command.system_time_ms=39 command.user_time_ms=25 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:30:56.709" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:26.809" grpc.time_ms=64.958 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2120gitlab # [ 255.816669] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.767Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"f590a2366d12573f473dabd3","created_at":"2026-09-06T16:30:25.404Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-dd664bbd722b405d","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:25.405Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-f590a2366d12573f473dabd3: done: 0.014819 sec","job_status":"done","queue_duration_s":1.34691,"scheduling_latency_s":1.346923,"redis_calls":7,"redis_duration_s":0.000891,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000654,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000237,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00975,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014819,"completed_at":"2026-09-06T16:30:26.767Z","load_balancing_strategy":"primary","db_duration_s":0.00243,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2121gitlab # [ 255.861207] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.768Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"52adb917838e692dee235155","created_at":"2026-09-06T16:30:25.449Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-dd664bbd722b405d","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:25.450Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-52adb917838e692dee235155: start","job_status":"start","queue_duration_s":1.317835,"scheduling_latency_s":1.317849}2122gitlab # [ 255.864921] gitaly[921]: time="2026-09-06T16:30:26.911Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:30:56.810" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:26.910" grpc.time_ms=0.637 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2123gitlab # [ 255.876704] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.942Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"52adb917838e692dee235155","created_at":"2026-09-06T16:30:25.449Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-dd664bbd722b405d","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:25.450Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-52adb917838e692dee235155: done: 0.174134 sec","job_status":"done","queue_duration_s":1.317835,"scheduling_latency_s":1.317849,"gitaly_calls":3,"gitaly_duration_s":0.072398,"redis_calls":63,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.007824,"redis_read_bytes":143,"redis_write_bytes":16719,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000322,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000677,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":20,"redis_queues_duration_s":0.001774,"redis_queues_read_bytes":23,"redis_queues_write_bytes":11225,"redis_queues_metadata_calls":18,"redis_queues_metadata_duration_s":0.002639,"redis_queues_metadata_read_bytes":28,"redis_queues_metadata_write_bytes":2886,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000149,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00082,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001121,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079046,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.153826534,"duration_s":0.174134,"completed_at":"2026-09-06T16:30:26.942Z","load_balancing_strategy":"primary","db_duration_s":0.019812,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2124gitlab # [ 255.954996] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.943Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","90ec96453585b61834523f2d"],"class":"WebHooks::LogExecutionWorker","jid":"7749493b8d53d414b5c50ebe","created_at":"2026-09-06T16:30:26.309Z","trace_propagation_headers":{"sentry-trace":"4f27504bc6814cdd815c0b8bfbca3e58-0c5544bb68024060","baggage":"sentry-trace_id=4f27504bc6814cdd815c0b8bfbca3e58,sentry-release=Unknown"},"correlation_id":"01M1VRWWYX5A86HQ0FEQBYK3DY","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:aa76f98bf1519327b8c6dfdd6159a3482f5aa98aa0a47c4f89c8c2e173bb0d15","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.312Z","job_size_bytes":2590,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-7749493b8d53d414b5c50ebe: start","job_status":"start","queue_duration_s":0.630921,"scheduling_latency_s":0.630935}2125gitlab # [ 255.955669] gitaly[921]: time="2026-09-06T16:30:26.934Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRVKT35ZTCG7ZKA5SS3SN6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:30:56.833" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:26.933" grpc.time_ms=0.552 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2126gitlab # [ 255.956479] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.960Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","90ec96453585b61834523f2d"],"class":"WebHooks::LogExecutionWorker","jid":"7749493b8d53d414b5c50ebe","created_at":"2026-09-06T16:30:26.309Z","trace_propagation_headers":{"sentry-trace":"4f27504bc6814cdd815c0b8bfbca3e58-0c5544bb68024060","baggage":"sentry-trace_id=4f27504bc6814cdd815c0b8bfbca3e58,sentry-release=Unknown"},"correlation_id":"01M1VRWWYX5A86HQ0FEQBYK3DY","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:aa76f98bf1519327b8c6dfdd6159a3482f5aa98aa0a47c4f89c8c2e173bb0d15","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.312Z","job_size_bytes":2590,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-7749493b8d53d414b5c50ebe: done: 0.016911 sec","job_status":"done","queue_duration_s":0.630921,"scheduling_latency_s":0.630935,"redis_calls":7,"redis_duration_s":0.000933,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000667,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000266,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011303,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016911,"completed_at":"2026-09-06T16:30:26.960Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002812,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2127gitlab # [ 256.012592] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:26.961Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"fbb3fa71c6aa1465f0d77730","created_at":"2026-09-06T16:30:26.348Z","trace_propagation_headers":{"sentry-trace":"0b62de9e8d014266b4c693d30fa5d97e-d32ada38f2914758","baggage":"sentry-trace_id=0b62de9e8d014266b4c693d30fa5d97e,sentry-release=Unknown"},"correlation_id":"01M1VRVF92VENKA2GZPJSCYPBZ","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.355Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-fbb3fa71c6aa1465f0d77730: start","job_status":"start","queue_duration_s":0.606396,"scheduling_latency_s":0.606409}2128gitlab # [ 256.013485] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:27.027Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"fbb3fa71c6aa1465f0d77730","created_at":"2026-09-06T16:30:26.348Z","trace_propagation_headers":{"sentry-trace":"0b62de9e8d014266b4c693d30fa5d97e-d32ada38f2914758","baggage":"sentry-trace_id=0b62de9e8d014266b4c693d30fa5d97e,sentry-release=Unknown"},"correlation_id":"01M1VRVF92VENKA2GZPJSCYPBZ","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.355Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-fbb3fa71c6aa1465f0d77730: done: 0.065418 sec","job_status":"done","queue_duration_s":0.606396,"scheduling_latency_s":0.606409,"redis_calls":11,"redis_duration_s":0.001585,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000243,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001055,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000287,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048295,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000232152,"duration_s":0.065418,"completed_at":"2026-09-06T16:30:27.027Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014332,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2129gitlab # [ 256.040326] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:27.028Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"a8877e3eb9b2968657bf9001","created_at":"2026-09-06T16:30:26.804Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-2fa206f19c134658","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.805Z","job_size_bytes":23,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-a8877e3eb9b2968657bf9001: start","job_status":"start","queue_duration_s":0.222503,"scheduling_latency_s":0.222517}2130gitlab # [ 256.054927] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:27.054Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"a8877e3eb9b2968657bf9001","created_at":"2026-09-06T16:30:26.804Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-2fa206f19c134658","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.805Z","job_size_bytes":23,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-a8877e3eb9b2968657bf9001: done: 0.025984 sec","job_status":"done","queue_duration_s":0.222503,"scheduling_latency_s":0.222517,"redis_calls":7,"redis_duration_s":0.0009500000000000001,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000702,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000248,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018868,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025984,"completed_at":"2026-09-06T16:30:27.054Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004484,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2131gitlab # [ 256.102636] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:27.055Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"c976b26115ec120d665d642e","created_at":"2026-09-06T16:30:26.879Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-2fa206f19c134658","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.881Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-c976b26115ec120d665d642e: start","job_status":"start","queue_duration_s":0.173531,"scheduling_latency_s":0.173545}2132gitlab # [ 256.107321] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:27.071Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"c976b26115ec120d665d642e","created_at":"2026-09-06T16:30:26.879Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-2fa206f19c134658","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.881Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-c976b26115ec120d665d642e: done: 0.015825 sec","job_status":"done","queue_duration_s":0.173531,"scheduling_latency_s":0.173545,"redis_calls":7,"redis_duration_s":0.000923,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000678,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000245,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010814,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015825,"completed_at":"2026-09-06T16:30:27.071Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002488,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2133gitlab # [ 256.163289] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:27.071Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"8f7261fd111dce25a2a1a177","created_at":"2026-09-06T16:30:26.887Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-2fa206f19c134658","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.888Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-8f7261fd111dce25a2a1a177: start","job_status":"start","queue_duration_s":0.18319,"scheduling_latency_s":0.183203}2134gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0'2135gitlab # [ 256.163929] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:27.089Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"8f7261fd111dce25a2a1a177","created_at":"2026-09-06T16:30:26.887Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-2fa206f19c134658","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.888Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-8f7261fd111dce25a2a1a177: done: 0.017231 sec","job_status":"done","queue_duration_s":0.18319,"scheduling_latency_s":0.183203,"redis_calls":7,"redis_duration_s":0.000951,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00066,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000291,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011401,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017231,"completed_at":"2026-09-06T16:30:27.089Z","load_balancing_strategy":"primary","db_duration_s":0.003601,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2136gitlab # [ 256.230748] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:27.090Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"d222b14e8aa3d4a549fc38a2","created_at":"2026-09-06T16:30:26.894Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-2fa206f19c134658","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.895Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-d222b14e8aa3d4a549fc38a2: start","job_status":"start","queue_duration_s":0.195139,"scheduling_latency_s":0.195153}2137gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0', in 0.04 seconds)2138gitlab # [ 256.231684] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:27.108Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"d222b14e8aa3d4a549fc38a2","created_at":"2026-09-06T16:30:26.894Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-2fa206f19c134658","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.895Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-d222b14e8aa3d4a549fc38a2: done: 0.018168 sec","job_status":"done","queue_duration_s":0.195139,"scheduling_latency_s":0.195153,"redis_calls":10,"redis_duration_s":0.00141,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00066,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000428,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000322,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012583,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018168,"completed_at":"2026-09-06T16:30:27.108Z","load_balancing_strategy":"primary","db_duration_s":0.003245,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2139gitlab # [ 256.270200] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:27.109Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"6dba1166641d339d410e2749","created_at":"2026-09-06T16:30:26.895Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-2fa206f19c134658","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.896Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-6dba1166641d339d410e2749: start","job_status":"start","queue_duration_s":0.212964,"scheduling_latency_s":0.212978}2140gitlab # [ 256.270894] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:27.125Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"6dba1166641d339d410e2749","created_at":"2026-09-06T16:30:26.895Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-2fa206f19c134658","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.896Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-6dba1166641d339d410e2749: done: 0.015528 sec","job_status":"done","queue_duration_s":0.212964,"scheduling_latency_s":0.212978,"redis_calls":7,"redis_duration_s":0.0009339999999999999,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000683,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000251,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015528,"completed_at":"2026-09-06T16:30:27.125Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002463,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2141gitlab # [ 256.309993] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:27.126Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"f84cb83733184b0987f07d67","created_at":"2026-09-06T16:30:26.897Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-2fa206f19c134658","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.898Z","job_size_bytes":19,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-f84cb83733184b0987f07d67: start","job_status":"start","queue_duration_s":0.227719,"scheduling_latency_s":0.227732}2142gitlab # [ 256.313943] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:27.140Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"f84cb83733184b0987f07d67","created_at":"2026-09-06T16:30:26.897Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-2fa206f19c134658","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.898Z","job_size_bytes":19,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-f84cb83733184b0987f07d67: done: 0.014299 sec","job_status":"done","queue_duration_s":0.227719,"scheduling_latency_s":0.227732,"redis_calls":8,"redis_duration_s":0.00104,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000803,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000237,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009573,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014299,"completed_at":"2026-09-06T16:30:27.140Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002063,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2143gitlab # [ 256.354716] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:27.141Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"78140fc1f3921a96804d25a2","created_at":"2026-09-06T16:30:26.899Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-2fa206f19c134658","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.900Z","job_size_bytes":66,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-78140fc1f3921a96804d25a2: start","job_status":"start","queue_duration_s":0.241199,"scheduling_latency_s":0.241213}2144gitlab # [ 256.368644] nixbot[2644]: INFO: ::ffff:127.0.0.1:37072 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 HTTP/1.1" 200 OK2145gitlab # [ 256.369800] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:27.158Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"78140fc1f3921a96804d25a2","created_at":"2026-09-06T16:30:26.899Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-2fa206f19c134658","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.900Z","job_size_bytes":66,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-78140fc1f3921a96804d25a2: done: 0.016869 sec","job_status":"done","queue_duration_s":0.241199,"scheduling_latency_s":0.241213,"redis_calls":7,"redis_duration_s":0.001107,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000827,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00028,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016869,"completed_at":"2026-09-06T16:30:27.158Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001849,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2146gitlab # [ 256.414460] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:27.159Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"caa6e10b7b41a4905284b8c9","created_at":"2026-09-06T16:30:26.900Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-2fa206f19c134658","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.901Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-caa6e10b7b41a4905284b8c9: start","job_status":"start","queue_duration_s":0.258107,"scheduling_latency_s":0.258159}2147gitlab # [ 256.424397] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:27.179Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"caa6e10b7b41a4905284b8c9","created_at":"2026-09-06T16:30:26.900Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-2fa206f19c134658","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.901Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-caa6e10b7b41a4905284b8c9: done: 0.019767 sec","job_status":"done","queue_duration_s":0.258107,"scheduling_latency_s":0.258159,"redis_calls":7,"redis_duration_s":0.001057,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000784,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000273,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012233,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019767,"completed_at":"2026-09-06T16:30:27.179Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004415,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2148gitlab # [ 256.480441] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:27.180Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"61488df94660bb95b6057005","created_at":"2026-09-06T16:30:26.901Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-2fa206f19c134658","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.902Z","job_size_bytes":6,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-61488df94660bb95b6057005: start","job_status":"start","queue_duration_s":0.277886,"scheduling_latency_s":0.2779}2149gitlab # [ 256.482188] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:27 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 11062150gitlab # [ 256.483529] gitaly[921]: time="2026-09-06T16:30:27.654Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1VRX3QNC1ZWV5S3AZ4W7G2Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:30:27.645" grpc.time_ms=9.317 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc2151gitlab # [ 256.492085] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:27.189Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"61488df94660bb95b6057005","created_at":"2026-09-06T16:30:26.901Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-2fa206f19c134658","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.902Z","job_size_bytes":6,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-61488df94660bb95b6057005: done: 0.009378 sec","job_status":"done","queue_duration_s":0.277886,"scheduling_latency_s":0.2779,"redis_calls":7,"redis_duration_s":0.001001,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000667,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000334,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009378,"completed_at":"2026-09-06T16:30:27.189Z","load_balancing_strategy":"primary","db_duration_s":0.000242,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2152gitlab # [ 256.536581] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:27.190Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"59584f0b75f7157cf90afa4a","created_at":"2026-09-06T16:30:26.902Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-2fa206f19c134658","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.904Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-59584f0b75f7157cf90afa4a: start","job_status":"start","queue_duration_s":0.286638,"scheduling_latency_s":0.286651}2153gitlab # [ 256.579862] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:27.205Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"59584f0b75f7157cf90afa4a","created_at":"2026-09-06T16:30:26.902Z","trace_propagation_headers":{"sentry-trace":"c50ddbebf7914b5d88a39b24f3503bb0-2fa206f19c134658","baggage":"sentry-trace_id=c50ddbebf7914b5d88a39b24f3503bb0,sentry-release=Unknown"},"correlation_id":"01M1VRVKT35ZTCG7ZKA5SS3SN6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:26.904Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-59584f0b75f7157cf90afa4a: done: 0.014383 sec","job_status":"done","queue_duration_s":0.286638,"scheduling_latency_s":0.286651,"redis_calls":7,"redis_duration_s":0.000996,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000677,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000319,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009512,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014383,"completed_at":"2026-09-06T16:30:27.205Z","load_balancing_strategy":"primary","db_duration_s":0.002232,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2154gitlab # [ 256.667296] gitaly[921]: time="2026-09-06T16:30:27.899Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX4TAX6JCPJZS75BHWNS2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:30:27.879" pid=921 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc2155gitlab # [ 256.668468] gitaly[921]: time="2026-09-06T16:30:27.899Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1170 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX4TAX6JCPJZS75BHWNS2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:30:27.879" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc2156gitlab # [ 256.669266] gitaly[921]: time="2026-09-06T16:30:27.900Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1170 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX4TAX6JCPJZS75BHWNS2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:30:27.879" grpc.time_ms=20.719 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc2157gitlab # [ 256.702365] gitlab-workhorse[826]: time="2026-09-06T16:30:27Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2162158gitlab # [ 256.704313] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:27 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 2772159gitlab # [ 256.823988] gitaly[921]: time="2026-09-06T16:30:28.056Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX5477MV1PCBK9M3E8D21 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:30:58.054" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:28.054" grpc.time_ms=1.061 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2160gitlab # [ 256.915308] gitaly[921]: time="2026-09-06T16:30:28.147Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M1VRX5477MV1PCBK9M3E8D21 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-06T16:30:58.133" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:28.133" grpc.time_ms=13.545 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2161gitlab # [ 256.956844] gitaly[921]: time="2026-09-06T16:30:28.188Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1VRX5477MV1PCBK9M3E8D21 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:30:38.017" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:28.178" grpc.time_ms=10.773 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2162gitlab # [ 257.023257] gitaly[921]: time="2026-09-06T16:30:28.255Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX5477MV1PCBK9M3E8D21 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-06T16:30:38.094" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:30:28.254" grpc.time_ms=0.389 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2163gitlab # [ 257.058546] gitaly[921]: time="2026-09-06T16:30:28.290Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX5477MV1PCBK9M3E8D21 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-06T16:30:38.118" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:28.279" grpc.time_ms=11.213 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2164gitlab # [ 257.131873] gitaly[921]: time="2026-09-06T16:30:28.364Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRX5477MV1PCBK9M3E8D21 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:30:28.363" grpc.time_ms=0.375 pid=921 span.kind=server system=grpc2165gitlab # [ 257.167543] gitaly[921]: time="2026-09-06T16:30:28.399Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRX5477MV1PCBK9M3E8D21 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:30:28.399" grpc.time_ms=0.498 pid=921 span.kind=server system=grpc2166gitlab # [ 257.179359] gitaly[921]: time="2026-09-06T16:30:28.411Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX5477MV1PCBK9M3E8D21 forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:30:38.157" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:30:28.318" operations="[repack_refs]" pid=921 ref=refs/keep-around/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 remote_ip="2001:db8:1::1" revision=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 span.kind=server system=grpc username=root2167gitlab # [ 257.180802] gitaly[921]: time="2026-09-06T16:30:28.412Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX5477MV1PCBK9M3E8D21 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:30:38.157" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:30:28.318" pid=921 ref=refs/keep-around/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 remote_ip="2001:db8:1::1" repository_info="{false {6 703 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-09-06 16:29:26.349894308 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" revision=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 span.kind=server system=grpc username=root2168gitlab # [ 257.204301] gitaly[921]: time="2026-09-06T16:30:28.436Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX5477MV1PCBK9M3E8D21 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:30:38.157" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:30:28.318" pid=921 ref=refs/keep-around/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 remote_ip="2001:db8:1::1" revision=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2169gitlab # [ 257.205492] gitaly[921]: time="2026-09-06T16:30:28.437Z" level=info msg="optimized repository" clean_stale_data_time=23.247927ms clean_work_trees_time="232.991µs" commit_graph_time="10.895µs" correlation_id=01M1VRX5477MV1PCBK9M3E8D21 gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="10.895µs" pid=921 prune_time="9.778µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="12.292µs" storage_name=default system=housekeeping total_time=23.801349ms2170gitlab # [ 257.206245] gitaly[921]: time="2026-09-06T16:30:28.437Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX5477MV1PCBK9M3E8D21 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:30:38.157" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:30:28.318" pid=921 ref=refs/keep-around/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 remote_ip="2001:db8:1::1" revision=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 span.kind=server system=grpc username=root2171gitlab # [ 257.237614] gitaly[921]: time="2026-09-06T16:30:28.469Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX5477MV1PCBK9M3E8D21 diskcache=9224dfde-b346-44f8-8a2d-8856f1300516 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:30:38.157" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:30:28.318" pid=921 ref=refs/keep-around/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 remote_ip="2001:db8:1::1" revision=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 span.kind=server system=grpc username=root2172gitlab # [ 257.238843] gitaly[921]: time="2026-09-06T16:30:28.470Z" level=info msg="finished unary call with code OK" command.count=6 command.cpu_time_ms=93 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=6623 command.oublock=8 command.real_time_ms=108 command.spawn_token_fork_ms=0 command.system_time_ms=52 command.user_time_ms=41 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX5477MV1PCBK9M3E8D21 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:30:38.157" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:30:28.318" grpc.time_ms=151.568 pid=921 ref=refs/keep-around/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 remote_ip="2001:db8:1::1" revision=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 span.kind=server system=grpc username=root2173gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0'2174gitlab # [ 257.265799] gitaly[921]: time="2026-09-06T16:30:28.498Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX5477MV1PCBK9M3E8D21 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:30:58.197" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:28.497" grpc.time_ms=0.465 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2175gitlab # [ 257.279250] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:28.510Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"55b114b69c6e67da9e7a84c4","created_at":"2026-09-06T16:30:28.503Z","trace_propagation_headers":{"sentry-trace":"c29bea6ce3744801a65ddb30ce9d89a5-b7fcb25d06f6470b","baggage":"sentry-trace_id=c29bea6ce3744801a65ddb30ce9d89a5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRX5477MV1PCBK9M3E8D21","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:28.508Z","job_size_bytes":65,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-55b114b69c6e67da9e7a84c4: start","job_status":"start","queue_duration_s":0.002427,"scheduling_latency_s":0.002482}2176gitlab # [ 257.316196] nixbot[2644]: INFO: ::ffff:127.0.0.1:37084 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 HTTP/1.1" 200 OK2177gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d5b04fed53c5e317aeaca026c0c710c04f7ba3b0', in 0.05 seconds)2178(finished: subtest: push delivers the webhook, in 52.72 seconds)2179subtest: eval and build post commit statuses2180gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0/statuses2181gitlab # [ 257.330508] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:28.562Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"55b114b69c6e67da9e7a84c4","created_at":"2026-09-06T16:30:28.503Z","trace_propagation_headers":{"sentry-trace":"c29bea6ce3744801a65ddb30ce9d89a5-b7fcb25d06f6470b","baggage":"sentry-trace_id=c29bea6ce3744801a65ddb30ce9d89a5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRX5477MV1PCBK9M3E8D21","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:28.508Z","job_size_bytes":65,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-55b114b69c6e67da9e7a84c4: done: 0.051901 sec","job_status":"done","queue_duration_s":0.002427,"scheduling_latency_s":0.002482,"redis_calls":7,"redis_duration_s":0.005628,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002438,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00319,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031787,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.051901,"completed_at":"2026-09-06T16:30:28.562Z","load_balancing_strategy":"primary","db_duration_s":0.007643,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2182gitlab # [ 257.332839] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:28.564Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"a9922b6f5e1b1b6243169ad7","created_at":"2026-09-06T16:30:28.509Z","trace_propagation_headers":{"sentry-trace":"c29bea6ce3744801a65ddb30ce9d89a5-b7fcb25d06f6470b","baggage":"sentry-trace_id=c29bea6ce3744801a65ddb30ce9d89a5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRX5477MV1PCBK9M3E8D21","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:28.511Z","job_size_bytes":65,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-a9922b6f5e1b1b6243169ad7: start","job_status":"start","queue_duration_s":0.053886,"scheduling_latency_s":0.053916}2183gitlab # [ 257.373543] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:28.605Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"a9922b6f5e1b1b6243169ad7","created_at":"2026-09-06T16:30:28.509Z","trace_propagation_headers":{"sentry-trace":"c29bea6ce3744801a65ddb30ce9d89a5-b7fcb25d06f6470b","baggage":"sentry-trace_id=c29bea6ce3744801a65ddb30ce9d89a5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRX5477MV1PCBK9M3E8D21","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:28.511Z","job_size_bytes":65,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-a9922b6f5e1b1b6243169ad7: done: 0.040856 sec","job_status":"done","queue_duration_s":0.053886,"scheduling_latency_s":0.053916,"redis_calls":10,"redis_duration_s":0.002501,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001641,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.00086,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024344,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040856,"completed_at":"2026-09-06T16:30:28.605Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008957,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2184gitlab # [ 257.414579] gitaly[921]: time="2026-09-06T16:30:28.632Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX5PZ8QBR341TZBPT65XK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:30:58.330" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:28.631" grpc.time_ms=0.616 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2185gitlab # [ 257.434037] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:28 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0/statuses HTTP/1.1" 200 2 "" "curl/8.21.0" 902186gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0/statuses, in 0.12 seconds)2187gitlab # [ 257.508813] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:28.740Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"3345ea00fddeac3f3e209531","created_at":"2026-09-06T16:30:28.735Z","trace_propagation_headers":{"sentry-trace":"c29bea6ce3744801a65ddb30ce9d89a5-b7fcb25d06f6470b","baggage":"sentry-trace_id=c29bea6ce3744801a65ddb30ce9d89a5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRX5477MV1PCBK9M3E8D21","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:28.739Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-3345ea00fddeac3f3e209531: start","job_status":"start","queue_duration_s":0.00157,"scheduling_latency_s":0.001601}2188gitlab # [ 257.561342] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:28 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 8172189gitlab # [ 257.568329] nixbot[2644]: {"ts": "2026-09-06T16:30:28.794Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 \"HTTP/1.1 201 Created\""}2190gitlab # [ 257.580180] nixbot[2644]: {"ts": "2026-09-06T16:30:28.812Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-d5b04fed53c5-5"}2191gitlab # [ 257.588530] gitaly[921]: time="2026-09-06T16:30:28.820Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VRX5477MV1PCBK9M3E8D21 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:30:58.718" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:28.819" grpc.time_ms=0.953 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2192gitlab # [ 257.619943] gitaly[921]: time="2026-09-06T16:30:28.852Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX5477MV1PCBK9M3E8D21 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:30:58.751" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:28.851" grpc.time_ms=0.603 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2193gitlab # [ 257.634576] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:28.859Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"3345ea00fddeac3f3e209531","created_at":"2026-09-06T16:30:28.735Z","trace_propagation_headers":{"sentry-trace":"c29bea6ce3744801a65ddb30ce9d89a5-b7fcb25d06f6470b","baggage":"sentry-trace_id=c29bea6ce3744801a65ddb30ce9d89a5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VRX5477MV1PCBK9M3E8D21","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:28.739Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-3345ea00fddeac3f3e209531: done: 0.119351 sec","job_status":"done","queue_duration_s":0.00157,"scheduling_latency_s":0.001601,"gitaly_calls":2,"gitaly_duration_s":0.007854,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.006797,"redis_read_bytes":79,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001292,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000784,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000506,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001395,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000334,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000334,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00086,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.081274,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.09325012,"duration_s":0.119351,"completed_at":"2026-09-06T16:30:28.859Z","load_balancing_strategy":"primary","db_duration_s":0.026665,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2194gitlab # [ 257.640631] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:28.860Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"1af9e97dc73caa29a51d2acd","created_at":"2026-09-06T16:30:28.797Z","trace_propagation_headers":{"sentry-trace":"c29bea6ce3744801a65ddb30ce9d89a5-bd971368c6434d0f","baggage":"sentry-trace_id=c29bea6ce3744801a65ddb30ce9d89a5,sentry-release=Unknown"},"correlation_id":"01M1VRX5477MV1PCBK9M3E8D21","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:28.799Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-1af9e97dc73caa29a51d2acd: start","job_status":"start","queue_duration_s":0.060974,"scheduling_latency_s":0.060988}2195gitlab # [ 257.691342] nixbot[2644]: {"ts": "2026-09-06T16:30:28.923Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-d5b04fed53c5-5", "argv0": "bwrap"}2196gitlab # [ 257.715448] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:28.947Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"1af9e97dc73caa29a51d2acd","created_at":"2026-09-06T16:30:28.797Z","trace_propagation_headers":{"sentry-trace":"c29bea6ce3744801a65ddb30ce9d89a5-bd971368c6434d0f","baggage":"sentry-trace_id=c29bea6ce3744801a65ddb30ce9d89a5,sentry-release=Unknown"},"correlation_id":"01M1VRX5477MV1PCBK9M3E8D21","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:28.799Z","job_size_bytes":3,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-1af9e97dc73caa29a51d2acd: done: 0.086591 sec","job_status":"done","queue_duration_s":0.060974,"scheduling_latency_s":0.060988,"redis_calls":7,"redis_duration_s":0.0011510000000000001,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000867,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000284,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.065,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.063,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018139,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.086591,"completed_at":"2026-09-06T16:30:28.947Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.065316,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2197gitlab # [ 257.793723] nix-daemon[2820]: accepted connection from pid 3695, user nixbot2198gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0/statuses2199gitlab # [ 258.547093] gitaly[921]: time="2026-09-06T16:30:29.779Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX6TCNXZDBW7T3GB0GFD2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:30:59.077" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:29.778" grpc.time_ms=0.905 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2200gitlab # [ 258.594451] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:29 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1172201gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0/statuses, in 0.16 seconds)2202gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0/statuses2203gitlab # [ 259.709415] gitaly[921]: time="2026-09-06T16:30:30.940Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX7YM99NT5XS46NRDZ0HY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:31:00.239" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:30.939" grpc.time_ms=1.004 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2204gitlab # [ 259.777401] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:31 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1402205gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0/statuses, in 0.18 seconds)2206gitlab # [ 260.414158] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:31.645Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckExportJobsWorker","jid":"58ca1453245fdd101b35fe64","created_at":"2026-09-06T16:30:31.627Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bfc17404ab3bfa324c2f650a5053e10b","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:c05136a1912f23c3950f9f6243170d655fcc26d1e5b96896d7e248d72f9039fe","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:31.641Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckExportJobsWorker JID-58ca1453245fdd101b35fe64: start","job_status":"start","queue_duration_s":0.003073,"scheduling_latency_s":0.003126}2207gitlab # [ 260.500477] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:31.732Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckExportJobsWorker","jid":"58ca1453245fdd101b35fe64","created_at":"2026-09-06T16:30:31.627Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bfc17404ab3bfa324c2f650a5053e10b","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:c05136a1912f23c3950f9f6243170d655fcc26d1e5b96896d7e248d72f9039fe","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:31.641Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckExportJobsWorker JID-58ca1453245fdd101b35fe64: done: 0.087215 sec","job_status":"done","queue_duration_s":0.003073,"scheduling_latency_s":0.003126,"redis_calls":10,"redis_duration_s":0.007811,"redis_read_bytes":9,"redis_write_bytes":995,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001795,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003063,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":518,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002953,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":192,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022884,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000522972,"duration_s":0.087215,"completed_at":"2026-09-06T16:30:31.732Z","load_balancing_strategy":"primary","db_duration_s":0.008611,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2208gitlab # [ 260.567556] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:31.799Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"505e4264a4b330e64f385929","created_at":"2026-09-06T16:30:31.675Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67ca838f639c99d0cb351d427706c194","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:31.692Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-505e4264a4b330e64f385929: start","job_status":"start","queue_duration_s":0.106887,"scheduling_latency_s":0.106934}2209gitlab # [ 260.605112] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:31.837Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2210gitlab # [ 260.627072] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:31.858Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"505e4264a4b330e64f385929","created_at":"2026-09-06T16:30:31.675Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67ca838f639c99d0cb351d427706c194","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:31.692Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-505e4264a4b330e64f385929: done: 0.059307 sec","job_status":"done","queue_duration_s":0.106887,"scheduling_latency_s":0.106934,"redis_calls":9,"redis_duration_s":0.015411000000000001,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004715,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008436,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00226,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018672,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.059307,"completed_at":"2026-09-06T16:30:31.858Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000756,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2211gitlab # [ 260.694969] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:31.926Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","jid":"1a7378d943bf80032bd1a9d6","created_at":"2026-09-06T16:30:31.707Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8fe180c969706680fcc214cbb341d4f0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6fd978bae1387dafc89025ea66dbcf41624e32b5e06562317011e023b1327155","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:31.795Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleBulkDeleteJobArtifactCronWorker JID-1a7378d943bf80032bd1a9d6: start","job_status":"start","queue_duration_s":0.131431,"scheduling_latency_s":0.131481}2212gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0/statuses2213gitlab # [ 260.801940] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:32.033Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","jid":"1a7378d943bf80032bd1a9d6","created_at":"2026-09-06T16:30:31.707Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8fe180c969706680fcc214cbb341d4f0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6fd978bae1387dafc89025ea66dbcf41624e32b5e06562317011e023b1327155","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:31.795Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleBulkDeleteJobArtifactCronWorker JID-1a7378d943bf80032bd1a9d6: done: 0.107278 sec","job_status":"done","queue_duration_s":0.131431,"scheduling_latency_s":0.131481,"redis_calls":24,"redis_duration_s":0.041033,"redis_read_bytes":25,"redis_write_bytes":5977,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.009394,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":704,"redis_queues_calls":2,"redis_queues_duration_s":0.000719,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3810,"redis_queues_metadata_calls":11,"redis_queues_metadata_duration_s":0.023639,"redis_queues_metadata_read_bytes":14,"redis_queues_metadata_write_bytes":894,"redis_shared_state_calls":7,"redis_shared_state_duration_s":0.007281,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":569,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034478,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002032661,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.max_buckets":5,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.available_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.occupied_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.stale_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.missing_count":5,"duration_s":0.107278,"completed_at":"2026-09-06T16:30:32.033Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003364,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2214gitlab # [ 260.876548] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:32.040Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"20273fb436d3ff823db825ab","created_at":"2026-09-06T16:30:31.805Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8718b9231310ae9ccc0448f0bdcdab5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:30:31.821Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-20273fb436d3ff823db825ab: start","job_status":"start","queue_duration_s":0.218397,"scheduling_latency_s":0.218442}2215gitlab # [ 260.877182] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:32.086Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2216gitlab # [ 260.877621] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:32.104Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"20273fb436d3ff823db825ab","created_at":"2026-09-06T16:30:31.805Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8718b9231310ae9ccc0448f0bdcdab5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:30:31.821Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-20273fb436d3ff823db825ab: done: 0.064268 sec","job_status":"done","queue_duration_s":0.218397,"scheduling_latency_s":0.218442,"redis_calls":9,"redis_duration_s":0.014161,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002268,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.010825,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001068,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013585,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.064268,"completed_at":"2026-09-06T16:30:32.104Z","load_balancing_strategy":"primary","db_duration_s":0.017384,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2217gitlab # [ 260.932412] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:32.164Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"465395ac622a29640c46d717","created_at":"2026-09-06T16:30:31.838Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0e13d96d49f1e0ca156b612deac4986","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:31.855Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-465395ac622a29640c46d717: start","job_status":"start","queue_duration_s":0.308394,"scheduling_latency_s":0.308434}2218gitlab # [ 260.959686] gitaly[921]: time="2026-09-06T16:30:32.191Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRX945RQE0M1BAF36N11S3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:31:02.189" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:32.190" grpc.time_ms=0.911 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2219gitlab # [ 260.998602] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:32 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1602220gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0/statuses, in 0.22 seconds)2221gitlab # [ 261.009192] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:32.241Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"465395ac622a29640c46d717","created_at":"2026-09-06T16:30:31.838Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0e13d96d49f1e0ca156b612deac4986","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:31.855Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-465395ac622a29640c46d717: done: 0.077087 sec","job_status":"done","queue_duration_s":0.308394,"scheduling_latency_s":0.308434,"redis_calls":9,"redis_duration_s":0.010545,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004218,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00281,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003517,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.077087,"completed_at":"2026-09-06T16:30:32.241Z","load_balancing_strategy":"primary","db_duration_s":0.02798,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2222gitlab # [ 261.013367] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:32.245Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"2ae898f8afdeb2fef0871803","created_at":"2026-09-06T16:30:31.927Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5cc2bfe9067ed0baf1c2bb61f94da943","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:31.939Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-2ae898f8afdeb2fef0871803: start","job_status":"start","queue_duration_s":0.305913,"scheduling_latency_s":0.305965}2223gitlab # [ 261.033172] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:32.265Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2224gitlab # [ 261.043832] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:32.275Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"2ae898f8afdeb2fef0871803","created_at":"2026-09-06T16:30:31.927Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5cc2bfe9067ed0baf1c2bb61f94da943","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:31.939Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-2ae898f8afdeb2fef0871803: done: 0.030533 sec","job_status":"done","queue_duration_s":0.305913,"scheduling_latency_s":0.305965,"redis_calls":9,"redis_duration_s":0.011234,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001026,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00916,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001048,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030533,"completed_at":"2026-09-06T16:30:32.275Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000801,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2225gitlab # [ 261.087074] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:32.279Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"587228d8479f9612e78a5185","created_at":"2026-09-06T16:30:31.958Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"72b3039aba2ee2bb5874c19bc13e75f4","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:31.968Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-587228d8479f9612e78a5185: start","job_status":"start","queue_duration_s":0.310645,"scheduling_latency_s":0.310663}2226gitlab # [ 261.824759] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.055Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"587228d8479f9612e78a5185","created_at":"2026-09-06T16:30:31.958Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"72b3039aba2ee2bb5874c19bc13e75f4","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:31.968Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-587228d8479f9612e78a5185: done: 0.775902 sec","job_status":"done","queue_duration_s":0.310645,"scheduling_latency_s":0.310663,"redis_calls":564,"redis_duration_s":0.46158200000000005,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001319,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001925,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.458338,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.358484,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.775902,"completed_at":"2026-09-06T16:30:33.055Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001191,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2227gitlab # [ 261.827913] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.057Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"c912afba3058e0bb829f6e06","created_at":"2026-09-06T16:30:31.977Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bcd89171df876860a16c1e8be4fe6b25","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:31.995Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-c912afba3058e0bb829f6e06: start","job_status":"start","queue_duration_s":1.061705,"scheduling_latency_s":1.061725}2228gitlab # [ 261.851055] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.072Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"c912afba3058e0bb829f6e06","created_at":"2026-09-06T16:30:31.977Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bcd89171df876860a16c1e8be4fe6b25","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:31.995Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-c912afba3058e0bb829f6e06: done: 0.015659 sec","job_status":"done","queue_duration_s":1.061705,"scheduling_latency_s":1.061725,"redis_calls":11,"redis_duration_s":0.00236,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000874,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001076,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00041,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00872,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015659,"completed_at":"2026-09-06T16:30:33.072Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001142,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2229gitlab # [ 261.910758] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.074Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"ffcd1c1708529e0bca8e118f","created_at":"2026-09-06T16:30:31.969Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-0b5dcb5b61084694","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"8fe180c969706680fcc214cbb341d4f0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.001Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-ffcd1c1708529e0bca8e118f: start","job_status":"start","queue_duration_s":1.07285,"scheduling_latency_s":1.072863}2230gitlab # [ 261.920871] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.097Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"ffcd1c1708529e0bca8e118f","created_at":"2026-09-06T16:30:31.969Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-0b5dcb5b61084694","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"8fe180c969706680fcc214cbb341d4f0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.001Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-ffcd1c1708529e0bca8e118f: done: 0.023219 sec","job_status":"done","queue_duration_s":1.07285,"scheduling_latency_s":1.072863,"redis_calls":18,"redis_duration_s":0.004049,"redis_read_bytes":39,"redis_write_bytes":2116,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000578,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001064,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":471,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.002407,"redis_shared_state_read_bytes":34,"redis_shared_state_write_bytes":1324,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011494,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000330768,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":3,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":3,"duration_s":0.023219,"completed_at":"2026-09-06T16:30:33.097Z","load_balancing_strategy":"primary","db_duration_s":0.005638,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2231gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0/statuses2232gitlab # [ 261.971981] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.098Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"ecb008d9be68f8c268c17789","created_at":"2026-09-06T16:30:31.969Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-0b5dcb5b61084694","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"8fe180c969706680fcc214cbb341d4f0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.001Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-ecb008d9be68f8c268c17789: start","job_status":"start","queue_duration_s":1.097202,"scheduling_latency_s":1.097237}2233gitlab # [ 261.973950] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.114Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"ecb008d9be68f8c268c17789","created_at":"2026-09-06T16:30:31.969Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-0b5dcb5b61084694","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"8fe180c969706680fcc214cbb341d4f0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.001Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-ecb008d9be68f8c268c17789: done: 0.015993 sec","job_status":"done","queue_duration_s":1.097202,"scheduling_latency_s":1.097237,"redis_calls":15,"redis_duration_s":0.003164,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001157,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.002007,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008729,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":4,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":4,"duration_s":0.015993,"completed_at":"2026-09-06T16:30:33.114Z","load_balancing_strategy":"primary","db_duration_s":0.002635,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2234gitlab # [ 262.016614] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.115Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"2ff990f036ec04a9150fc9bc","created_at":"2026-09-06T16:30:31.969Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-0b5dcb5b61084694","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"8fe180c969706680fcc214cbb341d4f0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.001Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-2ff990f036ec04a9150fc9bc: start","job_status":"start","queue_duration_s":1.114314,"scheduling_latency_s":1.114327}2235gitlab # [ 262.022881] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.131Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"2ff990f036ec04a9150fc9bc","created_at":"2026-09-06T16:30:31.969Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-0b5dcb5b61084694","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"8fe180c969706680fcc214cbb341d4f0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.001Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-2ff990f036ec04a9150fc9bc: done: 0.015621 sec","job_status":"done","queue_duration_s":1.114314,"scheduling_latency_s":1.114327,"redis_calls":15,"redis_duration_s":0.0029040000000000003,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.001981,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008804,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":2,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":2,"duration_s":0.015621,"completed_at":"2026-09-06T16:30:33.131Z","load_balancing_strategy":"primary","db_duration_s":0.002598,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2236gitlab # [ 262.065918] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.132Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"97fa7f45f17825c76186b126","created_at":"2026-09-06T16:30:31.969Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-0b5dcb5b61084694","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"8fe180c969706680fcc214cbb341d4f0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.001Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-97fa7f45f17825c76186b126: start","job_status":"start","queue_duration_s":1.131008,"scheduling_latency_s":1.131021}2237gitlab # [ 262.072900] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.147Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"97fa7f45f17825c76186b126","created_at":"2026-09-06T16:30:31.969Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-0b5dcb5b61084694","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"8fe180c969706680fcc214cbb341d4f0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.001Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-97fa7f45f17825c76186b126: done: 0.015518 sec","job_status":"done","queue_duration_s":1.131008,"scheduling_latency_s":1.131021,"redis_calls":15,"redis_duration_s":0.002766,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000834,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.001932,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008723,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":0,"duration_s":0.015518,"completed_at":"2026-09-06T16:30:33.147Z","load_balancing_strategy":"primary","db_duration_s":0.002592,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2238gitlab # [ 262.151204] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.148Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"18375353ac8874bca658b376","created_at":"2026-09-06T16:30:31.969Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-0b5dcb5b61084694","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"8fe180c969706680fcc214cbb341d4f0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.001Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-18375353ac8874bca658b376: start","job_status":"start","queue_duration_s":1.147631,"scheduling_latency_s":1.147644}2239gitlab # [ 262.152236] gitaly[921]: time="2026-09-06T16:30:33.320Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRXA9W9XWKJW4G18Q7JZTY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:31:03.018" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:33.319" grpc.time_ms=0.562 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2240gitlab # [ 262.153105] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:33 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 592241gitlab # [ 262.153536] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"18375353ac8874bca658b376","created_at":"2026-09-06T16:30:31.969Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-0b5dcb5b61084694","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"8fe180c969706680fcc214cbb341d4f0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.001Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-18375353ac8874bca658b376: done: 0.019563 sec","job_status":"done","queue_duration_s":1.147631,"scheduling_latency_s":1.147644,"redis_calls":15,"redis_duration_s":0.003385,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001083,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.002302,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012359,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":4,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":4,"duration_s":0.019563,"completed_at":"2026-09-06T16:30:33.168Z","load_balancing_strategy":"primary","db_duration_s":0.002688,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2242gitlab # [ 262.204166] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.169Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Export::PruneProjectExportJobsWorker","jid":"c170f6a6eda75f1729246df0","created_at":"2026-09-06T16:30:32.020Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9db1492bf24c8a138eec44f2a7b43b03","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b2f7045536fe38235224f4d98e515e25b0b0eea1aa1ae4b4ab9f99ab22c53b43","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.039Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Export::PruneProjectExportJobsWorker JID-c170f6a6eda75f1729246df0: start","job_status":"start","queue_duration_s":1.130038,"scheduling_latency_s":1.130051}2243gitlab # [ 262.204898] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.186Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Export::PruneProjectExportJobsWorker","jid":"c170f6a6eda75f1729246df0","created_at":"2026-09-06T16:30:32.020Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9db1492bf24c8a138eec44f2a7b43b03","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b2f7045536fe38235224f4d98e515e25b0b0eea1aa1ae4b4ab9f99ab22c53b43","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.039Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Export::PruneProjectExportJobsWorker JID-c170f6a6eda75f1729246df0: done: 0.016849 sec","job_status":"done","queue_duration_s":1.130038,"scheduling_latency_s":1.130051,"redis_calls":10,"redis_duration_s":0.0021219999999999998,"redis_read_bytes":9,"redis_write_bytes":1173,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000445,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001263,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000414,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":236,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009541,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000337752,"duration_s":0.016849,"completed_at":"2026-09-06T16:30:33.186Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002515,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2244gitlab # [ 262.262562] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.187Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"f53489ada90c198ac4b1008c","created_at":"2026-09-06T16:30:32.056Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"04a93a12426dfe9bc8b73e5c2aeb1010","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.070Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-f53489ada90c198ac4b1008c: start","job_status":"start","queue_duration_s":1.116892,"scheduling_latency_s":1.116905}2245gitlab # [ 262.263564] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.367Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"f53489ada90c198ac4b1008c","created_at":"2026-09-06T16:30:32.056Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"04a93a12426dfe9bc8b73e5c2aeb1010","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.070Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-f53489ada90c198ac4b1008c: done: 0.179499 sec","job_status":"done","queue_duration_s":1.116892,"scheduling_latency_s":1.116905,"redis_calls":13,"redis_duration_s":0.002656,"redis_read_bytes":11,"redis_write_bytes":1567,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000922,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000882,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000852,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.036,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.13273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000422959,"exclusive_lock_hold_duration_s":0.163603196,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.179499,"completed_at":"2026-09-06T16:30:33.367Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.052109,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2246gitlab # [ 262.319159] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.368Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"6c4ec4be8ce0ca49a1c39806","created_at":"2026-09-06T16:30:32.078Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"32bed6f2dca16407352655c811d9baf5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:30:32.087Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-6c4ec4be8ce0ca49a1c39806: start","job_status":"start","queue_duration_s":1.280333,"scheduling_latency_s":1.280347}2247gitlab # [ 262.319886] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.387Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"6c4ec4be8ce0ca49a1c39806","created_at":"2026-09-06T16:30:32.078Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"32bed6f2dca16407352655c811d9baf5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:30:32.087Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-6c4ec4be8ce0ca49a1c39806: done: 0.019422 sec","job_status":"done","queue_duration_s":1.280333,"scheduling_latency_s":1.280347,"redis_calls":9,"redis_duration_s":0.00123,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000268,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000696,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000266,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012523,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019422,"completed_at":"2026-09-06T16:30:33.387Z","load_balancing_strategy":"primary","db_duration_s":0.002276,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2248gitlab # [ 262.364415] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.388Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"ce416c10de9eb3b278d3a25e","created_at":"2026-09-06T16:30:32.100Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"226e531f99f8c9d4af3d6ab16b2eeda6","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.106Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-ce416c10de9eb3b278d3a25e: start","job_status":"start","queue_duration_s":1.281994,"scheduling_latency_s":1.282009}2249gitlab # [ 262.365183] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.402Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"ce416c10de9eb3b278d3a25e","created_at":"2026-09-06T16:30:32.100Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"226e531f99f8c9d4af3d6ab16b2eeda6","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.106Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-ce416c10de9eb3b278d3a25e: done: 0.013932 sec","job_status":"done","queue_duration_s":1.281994,"scheduling_latency_s":1.282009,"redis_calls":9,"redis_duration_s":0.0011979999999999998,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000249,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000695,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000254,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013932,"completed_at":"2026-09-06T16:30:33.402Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001326,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2250gitlab # [ 262.409036] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.403Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"ff3126879034e2a161394558","created_at":"2026-09-06T16:30:32.109Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dcac12abb683c148df7803f867289c3f","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.172Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-ff3126879034e2a161394558: start","job_status":"start","queue_duration_s":1.23114,"scheduling_latency_s":1.231153}2251gitlab # [ 262.409744] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.418Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"ff3126879034e2a161394558","created_at":"2026-09-06T16:30:32.109Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dcac12abb683c148df7803f867289c3f","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.172Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-ff3126879034e2a161394558: done: 0.0149 sec","job_status":"done","queue_duration_s":1.23114,"scheduling_latency_s":1.231153,"redis_calls":9,"redis_duration_s":0.001317,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000279,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000783,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009139,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0149,"completed_at":"2026-09-06T16:30:33.418Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001705,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2252gitlab # [ 262.460542] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.419Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"ccbe7fcd29763bb1b25d3ec6","created_at":"2026-09-06T16:30:32.190Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba502a0026cb19327ff95bc726592837","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.205Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-ccbe7fcd29763bb1b25d3ec6: start","job_status":"start","queue_duration_s":1.213582,"scheduling_latency_s":1.213595}2253gitlab # [ 262.461502] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.433Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"ccbe7fcd29763bb1b25d3ec6","created_at":"2026-09-06T16:30:32.190Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba502a0026cb19327ff95bc726592837","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.205Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-ccbe7fcd29763bb1b25d3ec6: done: 0.014229 sec","job_status":"done","queue_duration_s":1.213582,"scheduling_latency_s":1.213595,"redis_calls":11,"redis_duration_s":0.001589,"redis_read_bytes":9,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000252,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000761,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000576,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000314844,"exclusive_lock_hold_duration_s":0.002300572,"duration_s":0.014229,"completed_at":"2026-09-06T16:30:33.433Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001229,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2254gitlab # [ 262.489050] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.434Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"cd9465e3727a0a905e251606","created_at":"2026-09-06T16:30:32.212Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"af38bb68e86cb08f77c5426539fa987b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.219Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-cd9465e3727a0a905e251606: start","job_status":"start","queue_duration_s":1.215224,"scheduling_latency_s":1.215237}2255gitlab # [ 262.533353] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.445Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"cd9465e3727a0a905e251606","created_at":"2026-09-06T16:30:32.212Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"af38bb68e86cb08f77c5426539fa987b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.219Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-cd9465e3727a0a905e251606: done: 0.010831 sec","job_status":"done","queue_duration_s":1.215224,"scheduling_latency_s":1.215237,"redis_calls":11,"redis_duration_s":0.001382,"redis_read_bytes":10,"redis_write_bytes":1022,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000253,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":283,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000388,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":225,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006494,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000260369,"duration_s":0.010831,"completed_at":"2026-09-06T16:30:33.445Z","load_balancing_strategy":"primary","db_duration_s":0.000272,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2256gitlab # [ 262.575833] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.446Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"41886f04ce43af5e877e7ca4","created_at":"2026-09-06T16:30:32.224Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9aba2254c2d748fa8ef87ade35d214eb","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.234Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-41886f04ce43af5e877e7ca4: start","job_status":"start","queue_duration_s":1.211732,"scheduling_latency_s":1.211745}2257gitlab # [ 262.576747] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.465Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"41886f04ce43af5e877e7ca4","created_at":"2026-09-06T16:30:32.224Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9aba2254c2d748fa8ef87ade35d214eb","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.234Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-41886f04ce43af5e877e7ca4: done: 0.019181 sec","job_status":"done","queue_duration_s":1.211732,"scheduling_latency_s":1.211745,"redis_calls":20,"redis_duration_s":0.0024000000000000002,"redis_read_bytes":23,"redis_write_bytes":4315,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.000797,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1065,"redis_queues_calls":4,"redis_queues_duration_s":0.000349,"redis_queues_read_bytes":6,"redis_queues_write_bytes":2020,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.001015,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":1004,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000239,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000205054,"duration_s":0.019181,"completed_at":"2026-09-06T16:30:33.465Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000677,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2258gitlab # [ 262.623721] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.466Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"ae6e69338ab94040c0bed85c","created_at":"2026-09-06T16:30:32.242Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf05822d1802ab824b6d1b219debac9c","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.247Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-ae6e69338ab94040c0bed85c: start","job_status":"start","queue_duration_s":1.218681,"scheduling_latency_s":1.218695}2259gitlab # [ 262.624418] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.479Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"ae6e69338ab94040c0bed85c","created_at":"2026-09-06T16:30:32.242Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf05822d1802ab824b6d1b219debac9c","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.247Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-ae6e69338ab94040c0bed85c: done: 0.013347 sec","job_status":"done","queue_duration_s":1.218681,"scheduling_latency_s":1.218695,"redis_calls":10,"redis_duration_s":0.001312,"redis_read_bytes":9,"redis_write_bytes":1179,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00026,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000786,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000266,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":238,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008668,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000213435,"duration_s":0.013347,"completed_at":"2026-09-06T16:30:33.479Z","load_balancing_strategy":"primary","db_duration_s":0.000291,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2260gitlab # [ 262.665332] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.480Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"5972793785de00e5eadc0979","created_at":"2026-09-06T16:30:32.261Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8510e4476e13fff01983442672f1ff6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.269Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-5972793785de00e5eadc0979: start","job_status":"start","queue_duration_s":1.210616,"scheduling_latency_s":1.210628}2261gitlab # [ 262.666204] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.491Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"5972793785de00e5eadc0979","created_at":"2026-09-06T16:30:32.261Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8510e4476e13fff01983442672f1ff6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.269Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-5972793785de00e5eadc0979: done: 0.011231 sec","job_status":"done","queue_duration_s":1.210616,"scheduling_latency_s":1.210628,"redis_calls":9,"redis_duration_s":0.001149,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000251,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000657,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000241,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006967,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011231,"completed_at":"2026-09-06T16:30:33.491Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00026,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2262gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0/statuses, in 0.74 seconds)2263gitlab # [ 262.734869] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.492Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"1d4e03a95d8d0b82eec73de9","created_at":"2026-09-06T16:30:32.276Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b89344c6f3d7d32139a8238df96b0640","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.281Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-1d4e03a95d8d0b82eec73de9: start","job_status":"start","queue_duration_s":1.21104,"scheduling_latency_s":1.211053}2264gitlab # [ 262.747474] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.524Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"1d4e03a95d8d0b82eec73de9","created_at":"2026-09-06T16:30:32.276Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b89344c6f3d7d32139a8238df96b0640","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.281Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-1d4e03a95d8d0b82eec73de9: done: 0.03155 sec","job_status":"done","queue_duration_s":1.21104,"scheduling_latency_s":1.211053,"redis_calls":10,"redis_duration_s":0.001376,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00028,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000835,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000261,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01577,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000262045,"extra.pages_deactivate_expired_deployments_cron_worker.deactivate_expired_pages_deployments":{"deactivated_deployments":0,"duration":0.00150522},"duration_s":0.03155,"completed_at":"2026-09-06T16:30:33.523Z","load_balancing_strategy":"primary","db_duration_s":0.007126,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2265gitlab # [ 262.768712] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.525Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"0c99e5de953fc195ee63fa81","created_at":"2026-09-06T16:30:32.287Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e7875285f76c23ed9c1b73f4e408b11","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.295Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-0c99e5de953fc195ee63fa81: start","job_status":"start","queue_duration_s":1.229719,"scheduling_latency_s":1.229733}2266gitlab # [ 262.769710] nix-daemon[2820]: accepted connection from pid 3732, user nixbot2267gitlab # [ 262.770561] nix-daemon[2820]: accepted connection from pid 3695, user nixbot2268gitlab # [ 262.771153] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.536Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"0c99e5de953fc195ee63fa81","created_at":"2026-09-06T16:30:32.287Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e7875285f76c23ed9c1b73f4e408b11","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.295Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-0c99e5de953fc195ee63fa81: done: 0.011777 sec","job_status":"done","queue_duration_s":1.229719,"scheduling_latency_s":1.229733,"redis_calls":10,"redis_duration_s":0.001277,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00026,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000771,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000246,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000204496,"duration_s":0.011777,"completed_at":"2026-09-06T16:30:33.536Z","load_balancing_strategy":"primary","db_duration_s":0.000537,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2269gitlab # [ 262.798141] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.537Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"2eef07c4844e2ba736713a3d","created_at":"2026-09-06T16:30:32.302Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5ba6e166fe534515b4c9cf8f6f634e3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.307Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-2eef07c4844e2ba736713a3d: start","job_status":"start","queue_duration_s":1.230427,"scheduling_latency_s":1.230441}2270gitlab # [ 262.811690] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.544Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2271gitlab # [ 262.814918] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.548Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"2eef07c4844e2ba736713a3d","created_at":"2026-09-06T16:30:32.302Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5ba6e166fe534515b4c9cf8f6f634e3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.307Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-2eef07c4844e2ba736713a3d: done: 0.010766 sec","job_status":"done","queue_duration_s":1.230427,"scheduling_latency_s":1.230441,"redis_calls":9,"redis_duration_s":0.0011949999999999999,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000254,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000658,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000283,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006516,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010766,"completed_at":"2026-09-06T16:30:33.548Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00024,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2272gitlab # [ 262.865933] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.549Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"e416958e91261a7342e00017","created_at":"2026-09-06T16:30:32.377Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a3998af86c49311feb02ddcac7d22c84","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.395Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-e416958e91261a7342e00017: start","job_status":"start","queue_duration_s":1.153966,"scheduling_latency_s":1.153979}2273gitlab # [ 262.866777] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.565Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"e416958e91261a7342e00017","created_at":"2026-09-06T16:30:32.377Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a3998af86c49311feb02ddcac7d22c84","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.395Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-e416958e91261a7342e00017: done: 0.015777 sec","job_status":"done","queue_duration_s":1.153966,"scheduling_latency_s":1.153979,"redis_calls":10,"redis_duration_s":0.001346,"redis_read_bytes":9,"redis_write_bytes":1097,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000256,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":309,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000826,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000264,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":218,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000246959,"duration_s":0.015777,"completed_at":"2026-09-06T16:30:33.565Z","load_balancing_strategy":"primary","db_duration_s":0.00206,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2274gitlab # [ 262.924115] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.566Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"597e7312a33e0a34894185f6","created_at":"2026-09-06T16:30:32.409Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9df594be3f323eef867ab3c31f7b2dad","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.425Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-597e7312a33e0a34894185f6: start","job_status":"start","queue_duration_s":1.140823,"scheduling_latency_s":1.140836}2275gitlab # [ 262.924841] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.580Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"597e7312a33e0a34894185f6","created_at":"2026-09-06T16:30:32.409Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9df594be3f323eef867ab3c31f7b2dad","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.425Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-597e7312a33e0a34894185f6: done: 0.014114 sec","job_status":"done","queue_duration_s":1.140823,"scheduling_latency_s":1.140836,"redis_calls":9,"redis_duration_s":0.0012079999999999999,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000253,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000704,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000251,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008137,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014114,"completed_at":"2026-09-06T16:30:33.580Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002059,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2276gitlab # [ 262.978190] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.581Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"d5f43957c97151f26dab4cfd","created_at":"2026-09-06T16:30:32.435Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9cb1a61d2038a53bb554998315e57325","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.450Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-d5f43957c97151f26dab4cfd: start","job_status":"start","queue_duration_s":1.130766,"scheduling_latency_s":1.130779}2277gitlab # [ 262.978924] gitaly[921]: time="2026-09-06T16:30:34.148Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRXB3X53M7Y13187VYSP8D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:31:04.147" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:34.147" grpc.time_ms=0.503 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2278gitlab # [ 262.983671] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.605Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"d5f43957c97151f26dab4cfd","created_at":"2026-09-06T16:30:32.435Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9cb1a61d2038a53bb554998315e57325","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.450Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-d5f43957c97151f26dab4cfd: done: 0.024593 sec","job_status":"done","queue_duration_s":1.130766,"scheduling_latency_s":1.130779,"redis_calls":13,"redis_duration_s":0.0017850000000000001,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000531,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000648,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000606,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013995,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000363174,"exclusive_lock_hold_duration_s":0.011380218,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.024593,"completed_at":"2026-09-06T16:30:33.605Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006642,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2279gitlab # [ 263.030229] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.606Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"8622ae3fc56dad79d5824f1a","created_at":"2026-09-06T16:30:32.482Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f2994d2fc81d301b2c0704dca11a7900","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.492Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-8622ae3fc56dad79d5824f1a: start","job_status":"start","queue_duration_s":1.114096,"scheduling_latency_s":1.11411}2280gitlab # [ 263.038982] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.627Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"8622ae3fc56dad79d5824f1a","created_at":"2026-09-06T16:30:32.482Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f2994d2fc81d301b2c0704dca11a7900","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.492Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-8622ae3fc56dad79d5824f1a: done: 0.020912 sec","job_status":"done","queue_duration_s":1.114096,"scheduling_latency_s":1.11411,"redis_calls":13,"redis_duration_s":0.002078,"redis_read_bytes":10,"redis_write_bytes":1343,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000254,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000912,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":368,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013208,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000335797,"extra.packages_cleanup_delete_orphaned_dependencies_worker.last_processed_packages_dependency_id":0,"extra.packages_cleanup_delete_orphaned_dependencies_worker.deleted_rows_count":0,"duration_s":0.020912,"completed_at":"2026-09-06T16:30:33.627Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002124,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2281gitlab # [ 263.101790] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.629Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"575d61faf1ada2d7f9ba8f7d","created_at":"2026-09-06T16:30:32.513Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0f0c201bab7ea7e69ee9102ce8092d63","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.521Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-575d61faf1ada2d7f9ba8f7d: start","job_status":"start","queue_duration_s":1.107367,"scheduling_latency_s":1.107387}2282gitlab # [ 263.102694] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.658Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"575d61faf1ada2d7f9ba8f7d","created_at":"2026-09-06T16:30:32.513Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0f0c201bab7ea7e69ee9102ce8092d63","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.521Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-575d61faf1ada2d7f9ba8f7d: done: 0.029108 sec","job_status":"done","queue_duration_s":1.107367,"scheduling_latency_s":1.107387,"redis_calls":18,"redis_duration_s":0.002742,"redis_read_bytes":20,"redis_write_bytes":4702,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000598,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000197,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001493,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000454,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020924,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029108,"completed_at":"2026-09-06T16:30:33.658Z","load_balancing_strategy":"primary","db_duration_s":0.001638,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2283gitlab # [ 263.144886] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.659Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"a73d58cefb01d2dfd8a8a430","created_at":"2026-09-06T16:30:32.530Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1a5012f8838ada617c0c4e43633e43a","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.537Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-a73d58cefb01d2dfd8a8a430: start","job_status":"start","queue_duration_s":1.121644,"scheduling_latency_s":1.121661}2284gitlab # [ 263.145681] gitaly[921]: time="2026-09-06T16:30:34.316Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VRXB3X53M7Y13187VYSP8D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:30:44.150" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:34.311" grpc.time_ms=5.563 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2285gitlab # [ 263.193636] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.672Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"a73d58cefb01d2dfd8a8a430","created_at":"2026-09-06T16:30:32.530Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1a5012f8838ada617c0c4e43633e43a","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.537Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-a73d58cefb01d2dfd8a8a430: done: 0.013846 sec","job_status":"done","queue_duration_s":1.121644,"scheduling_latency_s":1.121661,"redis_calls":10,"redis_duration_s":0.001362,"redis_read_bytes":9,"redis_write_bytes":1087,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00026,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000846,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":566,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000256,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":216,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009102,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000224889,"duration_s":0.013846,"completed_at":"2026-09-06T16:30:33.672Z","load_balancing_strategy":"primary","db_duration_s":0.000318,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2286gitlab # [ 263.234869] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.673Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"43e3c98d972cf177b7985a2a","created_at":"2026-09-06T16:30:32.545Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6544783fcb4ed001442c5dbc79586ec0","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.553Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-43e3c98d972cf177b7985a2a: start","job_status":"start","queue_duration_s":1.120386,"scheduling_latency_s":1.120399}2287gitlab # [ 263.235933] nixbot[2644]: {"ts": "2026-09-06T16:30:34.359Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 \"HTTP/1.1 201 Created\""}2288gitlab # [ 263.236825] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:34 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 2482289gitlab # [ 263.237773] gitaly[921]: time="2026-09-06T16:30:34.407Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRXBBTDSJMV25E3QF1ZE8F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:31:04.106" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:34.406" grpc.time_ms=0.492 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2290gitlab # [ 263.238496] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.684Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"43e3c98d972cf177b7985a2a","created_at":"2026-09-06T16:30:32.545Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6544783fcb4ed001442c5dbc79586ec0","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.553Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-43e3c98d972cf177b7985a2a: done: 0.01106 sec","job_status":"done","queue_duration_s":1.120386,"scheduling_latency_s":1.120399,"redis_calls":11,"redis_duration_s":0.001469,"redis_read_bytes":8,"redis_write_bytes":1094,"redis_cache_calls":2,"redis_cache_duration_s":0.000287,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000247,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000675,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00026,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01106,"completed_at":"2026-09-06T16:30:33.684Z","load_balancing_strategy":"primary","db_duration_s":0.000248,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2291gitlab # [ 263.283331] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.685Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"fb06f5593f70193be3af07f4","created_at":"2026-09-06T16:30:32.562Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9aa3eefba6d34ba1c05a211ae415d8f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.570Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-fb06f5593f70193be3af07f4: start","job_status":"start","queue_duration_s":1.115042,"scheduling_latency_s":1.115064}2292gitlab # [ 263.283985] gitaly[921]: time="2026-09-06T16:30:34.437Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VRXBBTDSJMV25E3QF1ZE8F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:30:44.271" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:34.432" grpc.time_ms=5.612 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2293gitlab # [ 263.290912] nixbot[2644]: {"ts": "2026-09-06T16:30:34.480Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 \"HTTP/1.1 201 Created\""}2294gitlab # [ 263.293413] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:34 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1162295gitlab # [ 263.295785] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.701Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"fb06f5593f70193be3af07f4","created_at":"2026-09-06T16:30:32.562Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9aa3eefba6d34ba1c05a211ae415d8f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.570Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-fb06f5593f70193be3af07f4: done: 0.015771 sec","job_status":"done","queue_duration_s":1.115042,"scheduling_latency_s":1.115064,"redis_calls":11,"redis_duration_s":0.001876,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000254,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00075,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000872,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000497269,"exclusive_lock_hold_duration_s":0.004341334,"duration_s":0.015771,"completed_at":"2026-09-06T16:30:33.701Z","load_balancing_strategy":"primary","db_duration_s":0.002476,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2296gitlab # [ 263.343585] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.702Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"e4132765a43e884c9457db18","created_at":"2026-09-06T16:30:32.580Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"16eb9767cd5cecaaeb448f0ff5779b7a","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.593Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-e4132765a43e884c9457db18: start","job_status":"start","queue_duration_s":1.108686,"scheduling_latency_s":1.1087}2297gitlab # [ 263.347677] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.714Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"e4132765a43e884c9457db18","created_at":"2026-09-06T16:30:32.580Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"16eb9767cd5cecaaeb448f0ff5779b7a","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.593Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-e4132765a43e884c9457db18: done: 0.012012 sec","job_status":"done","queue_duration_s":1.108686,"scheduling_latency_s":1.1087,"redis_calls":11,"redis_duration_s":0.001744,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000332,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000867,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000195,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00035,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000211479,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.012012,"completed_at":"2026-09-06T16:30:33.714Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000316,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2298gitlab # [ 263.405749] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.715Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"e27ab3603d228ab0cb645f3c","created_at":"2026-09-06T16:30:32.606Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"035fc2ee9d0f545bd2c57b8762ae7ee3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.618Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-e27ab3603d228ab0cb645f3c: start","job_status":"start","queue_duration_s":1.096878,"scheduling_latency_s":1.09689}2299gitlab # [ 263.406610] gitaly[921]: time="2026-09-06T16:30:34.538Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRXBFTZFQV4BCWGGYD265N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:31:04.236" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:30:34.537" grpc.time_ms=0.615 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2300gitlab # [ 263.414090] nixbot[2644]: {"ts": "2026-09-06T16:30:34.608Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 \"HTTP/1.1 201 Created\""}2301gitlab # [ 263.416731] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.778Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"e27ab3603d228ab0cb645f3c","created_at":"2026-09-06T16:30:32.606Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"035fc2ee9d0f545bd2c57b8762ae7ee3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.618Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-e27ab3603d228ab0cb645f3c: done: 0.062647 sec","job_status":"done","queue_duration_s":1.096878,"scheduling_latency_s":1.09689,"redis_calls":70,"redis_duration_s":0.010861999999999998,"redis_read_bytes":118,"redis_write_bytes":42480,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001615,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000537,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37385,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.008176,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000534,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050867,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.062647,"completed_at":"2026-09-06T16:30:33.778Z","load_balancing_strategy":"primary","db_duration_s":0.001267,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2302gitlab # [ 263.480768] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.779Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"4865fae9334b34ff5d7ceb7d","created_at":"2026-09-06T16:30:32.628Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"58cefc3ae8c92d89994d815f6cf2eb7b","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.643Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-4865fae9334b34ff5d7ceb7d: start","job_status":"start","queue_duration_s":1.135449,"scheduling_latency_s":1.135467}2303gitlab # [ 263.481712] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:34 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1172304gitlab # [ 263.484150] gitaly[921]: time="2026-09-06T16:30:34.571Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VRXBFTZFQV4BCWGGYD265N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:30:44.114" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:34.565" grpc.time_ms=5.666 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2305gitlab # [ 263.486156] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.799Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"4865fae9334b34ff5d7ceb7d","created_at":"2026-09-06T16:30:32.628Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"58cefc3ae8c92d89994d815f6cf2eb7b","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.643Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-4865fae9334b34ff5d7ceb7d: done: 0.019735 sec","job_status":"done","queue_duration_s":1.135449,"scheduling_latency_s":1.135467,"redis_calls":12,"redis_duration_s":0.002881,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000496,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001361,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001024,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012206,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.005170769,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.019735,"completed_at":"2026-09-06T16:30:33.799Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001902,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2306gitlab # [ 263.527899] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.800Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"017dcdbf866f33ad7b46a63b","created_at":"2026-09-06T16:30:32.667Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ea32ecf98c5c8f0f3739333ba593c5c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-06T16:30:32.679Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-017dcdbf866f33ad7b46a63b: start","job_status":"start","queue_duration_s":1.120376,"scheduling_latency_s":1.12039}2307gitlab # [ 263.576674] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.807Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2308gitlab # [ 263.577500] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.812Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"017dcdbf866f33ad7b46a63b","created_at":"2026-09-06T16:30:32.667Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ea32ecf98c5c8f0f3739333ba593c5c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-06T16:30:32.679Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-017dcdbf866f33ad7b46a63b: done: 0.011828 sec","job_status":"done","queue_duration_s":1.120376,"scheduling_latency_s":1.12039,"redis_calls":9,"redis_duration_s":0.002036,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000489,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001135,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000412,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006507,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011828,"completed_at":"2026-09-06T16:30:33.812Z","load_balancing_strategy":"primary","db_duration_s":0.000276,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2309gitlab # [ 263.581617] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.813Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"6be737d7d86f7aca6bf2eafa","created_at":"2026-09-06T16:30:32.690Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfe9b345c3545c5c30591abd57d1fbc6","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.702Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-6be737d7d86f7aca6bf2eafa: start","job_status":"start","queue_duration_s":1.111278,"scheduling_latency_s":1.111291}2310gitlab # [ 263.627151] gitaly[921]: time="2026-09-06T16:30:34.808Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRXB3X53M7Y13187VYSP8D grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:30:34.808" grpc.time_ms=0.295 pid=921 span.kind=server system=grpc2311gitlab # [ 263.627889] gitaly[921]: time="2026-09-06T16:30:34.833Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRXB3X53M7Y13187VYSP8D grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:30:34.833" grpc.time_ms=0.24 pid=921 span.kind=server system=grpc2312gitlab # [ 263.628652] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.830Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"6be737d7d86f7aca6bf2eafa","created_at":"2026-09-06T16:30:32.690Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfe9b345c3545c5c30591abd57d1fbc6","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.702Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-6be737d7d86f7aca6bf2eafa: done: 0.017276 sec","job_status":"done","queue_duration_s":1.111278,"scheduling_latency_s":1.111291,"redis_calls":10,"redis_duration_s":0.002508,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000478,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001622,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000408,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009488,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000309536,"duration_s":0.017276,"completed_at":"2026-09-06T16:30:33.830Z","load_balancing_strategy":"primary","db_duration_s":0.002407,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2313gitlab # [ 263.651943] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.831Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"68d64bb025fa9b53e93ff589","created_at":"2026-09-06T16:30:32.715Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"236d91281678321b8c7e777ff161b6d5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.726Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-68d64bb025fa9b53e93ff589: start","job_status":"start","queue_duration_s":1.104785,"scheduling_latency_s":1.104798}2314gitlab # [ 263.696808] gitaly[921]: time="2026-09-06T16:30:34.851Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VRXB3X53M7Y13187VYSP8D grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:30:34.851" grpc.time_ms=0.254 pid=921 span.kind=server system=grpc2315gitlab # [ 263.697683] gitaly[921]: time="2026-09-06T16:30:34.865Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VRXB3X53M7Y13187VYSP8D diskcache=8d784639-e454-48d9-9499-e036e2db433a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:31:04.686" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:34.788" pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2316gitlab # [ 263.698481] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:34 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 612317gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0/statuses2318gitlab # [ 263.698963] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.863Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"68d64bb025fa9b53e93ff589","created_at":"2026-09-06T16:30:32.715Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"236d91281678321b8c7e777ff161b6d5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.726Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-68d64bb025fa9b53e93ff589: done: 0.031683 sec","job_status":"done","queue_duration_s":1.104785,"scheduling_latency_s":1.104798,"redis_calls":11,"redis_duration_s":0.0031149999999999997,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000423,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001369,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001323,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000450336,"exclusive_lock_hold_duration_s":0.008242947,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.031683,"completed_at":"2026-09-06T16:30:33.863Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007909,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2319gitlab # [ 263.740812] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.864Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Import::ImportFileCleanupWorker","jid":"a9114c4b655b3765c26fcbcd","created_at":"2026-09-06T16:30:32.741Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f2e8a847a031a3a3afe2c30cf25138fc","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:872203d04eabf30611038084c617b8ff07fd3e5607b1d8e2c977d615a3cfd3cb","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.752Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Import::ImportFileCleanupWorker JID-a9114c4b655b3765c26fcbcd: start","job_status":"start","queue_duration_s":1.112081,"scheduling_latency_s":1.112111}2320gitlab # [ 263.741732] gitaly[921]: time="2026-09-06T16:30:34.865Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=62 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7409 command.oublock=0 command.real_time_ms=67 command.spawn_token_fork_ms=0 command.system_time_ms=40 command.user_time_ms=22 component=gitaly.UnaryServerInterceptor correlation_id=01M1VRXB3X53M7Y13187VYSP8D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:31:04.686" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:30:34.788" grpc.time_ms=76.787 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2321gitlab # [ 263.788646] gitlab-workhorse[826]: time="2026-09-06T16:30:34Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672322gitlab # [ 263.788831] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:30:34 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 562323gitlab # [ 263.789420] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.883Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Import::ImportFileCleanupWorker","jid":"a9114c4b655b3765c26fcbcd","created_at":"2026-09-06T16:30:32.741Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f2e8a847a031a3a3afe2c30cf25138fc","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:872203d04eabf30611038084c617b8ff07fd3e5607b1d8e2c977d615a3cfd3cb","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.752Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Import::ImportFileCleanupWorker JID-a9114c4b655b3765c26fcbcd: done: 0.019164 sec","job_status":"done","queue_duration_s":1.112081,"scheduling_latency_s":1.112111,"redis_calls":10,"redis_duration_s":0.001496,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000381,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000852,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000263,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011235,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000252546,"duration_s":0.019164,"completed_at":"2026-09-06T16:30:33.883Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002735,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2324gitlab # [ 263.791112] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.884Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"7a0a9bcf2a706bda3bd8ed14","created_at":"2026-09-06T16:30:32.765Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"178231c005786476bdd9b12cbf2d6696","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.777Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-7a0a9bcf2a706bda3bd8ed14: start","job_status":"start","queue_duration_s":1.107163,"scheduling_latency_s":1.107178}2325gitlab # [ 263.804265] gitaly[921]: time="2026-09-06T16:30:34.870Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VRXBSSZJB83548Z6M9AENJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:30:34.863" grpc.time_ms=7.149 pid=921 remote_ip="2001:db8:1::1" span.kind=server system=grpc2326gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d5b04fed53c5e317aeaca026c0c710c04f7ba3b0/statuses, in 0.12 seconds)2327(finished: subtest: eval and build post commit statuses, in 6.54 seconds)2328(finished: run the VM test script, in 311.84 seconds)2329test script finished in 311.86s2330cleanup2331kill QemuMachine (pid 46)2332gitlab # [ 263.849646] sidekiq-cluster[2536]: {"severity":"INFO","time":"2026-09-06T16:30:33.898Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"7a0a9bcf2a706bda3bd8ed14","created_at":"2026-09-06T16:30:32.765Z","trace_propagation_headers":{"sentry-trace":"d82f06e6f4364098b59555730680de3e-dfdb3089cf14482e","baggage":"sentry-trace_id=d82f06e6f4364098b59555730680de3e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"178231c005786476bdd9b12cbf2d6696","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:30:32.777Z","job_size_bytes":2,"pid":2536,"sidekiq_tid":"m7y4","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-7a0a9bcf2a706bda3bd8ed14: done: 0.013673 sec","job_status":"done","queue_duration_s":1.107163,"scheduling_latency_s":1.107178,"redis_calls":11,"redis_duration_s":0.001608,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000328,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00068,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.0006,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14)2333gitlab # ,2334(finished: cleanup, in 1.48 seconds)2335additionally exposed symbols:2336 gitlab,2337 vlan1,2338 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2339[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-06T16:29:35.506Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2340[]2341[]2342[]2343[]2344[]2345[]2346[]2347[]2348[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'd5b04fed53c5e317aeaca026c0c710c04f7ba3b0', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-09-06T16:30:27.967428Z', 'started_at': None, 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': {'position': 1, 'blocked_by': []}}]2349[]2350[{'id': 5, 'sha': 'd5b04fed53c5e317aeaca026c0c710c04f7ba3b0', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:30:28.728Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2351[{'id': 5, 'sha': 'd5b04fed53c5e317aeaca026c0c710c04f7ba3b0', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:30:28.728Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2352[{'id': 5, 'sha': 'd5b04fed53c5e317aeaca026c0c710c04f7ba3b0', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:30:28.728Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2353[{'id': 5, 'sha': 'd5b04fed53c5e317aeaca026c0c710c04f7ba3b0', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:30:28.728Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2354[{'id': 5, 'sha': 'd5b04fed53c5e317aeaca026c0c710c04f7ba3b0', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 5s (1 warning)', 'created_at': '2026-09-06T16:30:28.728Z', 'started_at': None, 'finished_at': '2026-09-06T16:30:34.333Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'd5b04fed53c5e317aeaca026c0c710c04f7ba3b0', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-09-06T16:30:34.458Z', 'started_at': None, 'finished_at': '2026-09-06T16:30:34.587Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]