vm-test-run-nixbot-gitlab
checks.x86_64-linux.nixbot-gitlab
· build #293
· raw
1Machine state will be reset. To keep it, pass --keep-machine-state2start all VLans3(finished: start all VLans, in 0.00 seconds)4Test will time out and terminate in 3600 seconds5run the VM test script6additionally exposed symbols:7 gitlab,8 vlan1,9 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh10start all VMs11gitlab: starting vm12gitlab # Disk image does not exist, creating the virtualisation disk image...13gitlab # Formatting '/build/vm-state-gitlab/tmp.xJxXwF7jdX', fmt=raw size=107374182414gitlab # mke2fs 1.47.4 (6-Mar-2025)15gitlab # Discarding device blocks: 0/262144 done16gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes17gitlab # Filesystem UUID: 91db64c9-2ec4-48f0-9355-e89f85a2574618gitlab # Superblock backups stored on blocks:19gitlab # 32768, 98304, 163840, 22937620gitlab # 21gitlab # Allocating group tables: 0/8 done22gitlab: QEMU running (pid 46)23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30(finished: start all VMs, in 61.50 seconds)31gitlab: waiting for unit setup-gitlab.service32gitlab: waiting for the VM to finish booting33gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)34gitlab # 35gitlab # 36gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA0037gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...38gitlab # 39gitlab # 40gitlab # 41gitlab # 42gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB0043gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...44gitlab # 45gitlab # 46gitlab # Booting from ROM...47gitlab # Probing EDD (edd=off to disable)... ok48gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 202649gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/2lzp8hvaxzcmd3b3cpwanxzjxxlviix8-nixos-system-gitlab-test/init regInfo=/nix/store/1svjzgj695xgfnla3rb3vmh0c65013dx-closure-info/registration console=ttyS0,115200n8 console=tty050gitlab # [ 0.000000] x86 CPU feature dependency check failure: CPU0 has '18*32+31' enabled but '18*32+26' disabled. Kernel might be fine, but no guarantees.51gitlab # [ 0.000000] BIOS-provided physical RAM map:52gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable53gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved54gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved55gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable56gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved57gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved58gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved59gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable60gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved61gitlab # [ 0.000000] NX (Execute Disable) protection: active62gitlab # [ 0.000000] APIC: Static calls initialized63gitlab # [ 0.000000] SMBIOS 2.8 present.64gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/201465gitlab # [ 0.000000] DMI: Memory slots populated: 1/166gitlab # [ 0.000000] Hypervisor detected: KVM67gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x40000000068gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d0069gitlab # [ 0.000001] kvm-clock: using sched offset of 597582923 cycles70gitlab # [ 0.000005] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns71gitlab # [ 0.000010] tsc: Detected 3099.998 MHz processor72gitlab # [ 0.000833] last_pfn = 0x1c0000 max_arch_pfn = 0x40000000073gitlab # [ 0.000863] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs74gitlab # [ 0.000866] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT75gitlab # [ 0.000914] last_pfn = 0xbffd8 max_arch_pfn = 0x40000000076gitlab # [ 0.003260] found SMP MP-table at [mem 0x000f5470-0x000f547f]77gitlab # [ 0.003286] Using GB pages for direct mapping78gitlab # [ 0.003477] RAMDISK: [mem 0xbe31c000-0xbffcffff]79gitlab # [ 0.003486] ACPI: Early table checksum verification disabled80gitlab # [ 0.003491] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )81gitlab # [ 0.003495] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)82gitlab # [ 0.003501] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)83gitlab # [ 0.003507] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001)84gitlab # [ 0.003509] ACPI: FACS 0x00000000BFFE0000 00004085gitlab # [ 0.003511] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001)86gitlab # [ 0.003513] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)87gitlab # [ 0.003515] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)88gitlab # [ 0.003517] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]89gitlab # [ 0.003519] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]90gitlab # [ 0.003519] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]91gitlab # [ 0.003520] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]92gitlab # [ 0.003521] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]93gitlab # [ 0.003521] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]94gitlab # [ 0.003968] No NUMA configuration found95gitlab # [ 0.003970] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]96gitlab # [ 0.003973] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff]97gitlab # [ 0.004087] Zone ranges:98gitlab # [ 0.004088] DMA [mem 0x0000000000001000-0x0000000000ffffff]99gitlab # [ 0.004089] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]100gitlab # [ 0.004090] Normal [mem 0x0000000100000000-0x00000001bfffffff]101gitlab # [ 0.004091] Device empty102gitlab # [ 0.004092] Movable zone start for each node103gitlab # [ 0.004093] Early memory node ranges104gitlab # [ 0.004093] node 0: [mem 0x0000000000001000-0x000000000009efff]105gitlab # [ 0.004094] node 0: [mem 0x0000000000100000-0x00000000bffd7fff]106gitlab # [ 0.004095] node 0: [mem 0x0000000100000000-0x00000001bfffffff]107gitlab # [ 0.004096] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]108gitlab # [ 0.004116] On node 0, zone DMA: 1 pages in unavailable ranges109gitlab # [ 0.004406] On node 0, zone DMA: 97 pages in unavailable ranges110gitlab # [ 0.122376] On node 0, zone Normal: 40 pages in unavailable ranges111gitlab # [ 0.123251] ACPI: PM-Timer IO Port: 0x608112gitlab # [ 0.123270] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])113gitlab # [ 0.123306] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23114gitlab # [ 0.123309] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)115gitlab # [ 0.123310] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)116gitlab # [ 0.123312] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)117gitlab # [ 0.123314] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)118gitlab # [ 0.123315] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)119gitlab # [ 0.123319] ACPI: Using ACPI (MADT) for SMP configuration information120gitlab # [ 0.123320] ACPI: HPET id: 0x8086a201 base: 0xfed00000121gitlab # [ 0.123327] TSC deadline timer available122gitlab # [ 0.123332] CPU topo: Max. logical packages: 1123gitlab # [ 0.123333] CPU topo: Max. logical dies: 1124gitlab # [ 0.123334] CPU topo: Max. dies per package: 1125gitlab # [ 0.123337] CPU topo: Max. threads per core: 1126gitlab # [ 0.123338] CPU topo: Num. cores per package: 4127gitlab # [ 0.123339] CPU topo: Num. threads per package: 4128gitlab # [ 0.123339] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs129gitlab # [ 0.123362] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()130gitlab # [ 0.123378] kvm-guest: KVM setup pv remote TLB flush131gitlab # [ 0.123381] kvm-guest: setup PV sched yield132gitlab # [ 0.123402] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]133gitlab # [ 0.123404] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]134gitlab # [ 0.123405] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]135gitlab # [ 0.123407] [mem 0xc0000000-0xfeffbfff] available for PCI devices136gitlab # [ 0.123408] Booting paravirtualized kernel on KVM137gitlab # [ 0.123413] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns138gitlab # [ 0.128165] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1139gitlab # [ 0.130538] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288140gitlab # [ 0.130593] kvm-guest: PV spinlocks enabled141gitlab # [ 0.130595] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)142gitlab # [ 0.130599] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/2lzp8hvaxzcmd3b3cpwanxzjxxlviix8-nixos-system-gitlab-test/init regInfo=/nix/store/1svjzgj695xgfnla3rb3vmh0c65013dx-closure-info/registration console=ttyS0,115200n8 console=tty0143gitlab # [ 0.130719] Unknown kernel command line parameters "regInfo=/nix/store/1svjzgj695xgfnla3rb3vmh0c65013dx-closure-info/registration", will be passed to user space.144gitlab # [ 0.130740] random: crng init done145gitlab # [ 0.130741] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes146gitlab # [ 0.141572] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)147gitlab # [ 0.146760] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)148gitlab # [ 0.146879] software IO TLB: area num 4.149gitlab # [ 0.236138] Fallback order for Node 0: 0150gitlab # [ 0.236148] Built 1 zonelists, mobility grouping on. Total pages: 1572726151gitlab # [ 0.236151] Policy zone: Normal152gitlab # [ 0.239119] mem auto-init: stack:all(zero), heap alloc:on, heap free:off153gitlab # [ 0.256234] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1154gitlab # [ 0.270699] allocated 12582912 bytes of page_ext155gitlab # [ 0.281282] ftrace: allocating 48717 entries in 192 pages156gitlab # [ 0.281287] ftrace: allocated 192 pages with 2 groups157gitlab # [ 0.282292] Dynamic Preempt: lazy158gitlab # [ 0.282511] rcu: Preemptible hierarchical RCU implementation.159gitlab # [ 0.282513] rcu: RCU event tracing is enabled.160gitlab # [ 0.282513] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.161gitlab # [ 0.282514] Trampoline variant of Tasks RCU enabled.162gitlab # [ 0.282515] Rude variant of Tasks RCU enabled.163gitlab # [ 0.282515] Tracing variant of Tasks RCU enabled.164gitlab # [ 0.282516] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.165gitlab # [ 0.282517] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4166gitlab # [ 0.282534] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.167gitlab # [ 0.282536] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.168gitlab # [ 0.282537] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.169gitlab # [ 0.288521] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16170gitlab # [ 0.288836] rcu: srcu_init: Setting srcu_struct sizes based on contention.171gitlab # [ 0.288848] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns172gitlab # [ 0.288981] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)173gitlab # [ 0.296015] Console: colour VGA+ 80x25174gitlab # [ 0.296022] printk: legacy console [tty0] enabled175gitlab # [ 0.341701] printk: legacy console [ttyS0] enabled176gitlab # [ 0.512473] ACPI: Core revision 20250807177gitlab # [ 0.513974] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns178gitlab # [ 0.516477] APIC: Switch to symmetric I/O mode setup179gitlab # [ 0.518149] x2apic enabled180gitlab # [ 0.519324] APIC: Switched APIC routing to: physical x2apic181gitlab # [ 0.521030] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()182gitlab # [ 0.523159] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()183gitlab # [ 0.525771] kvm-guest: setup PV IPIs184gitlab # [ 0.527884] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1185gitlab # [ 0.529411] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x2caf45e6a57, max_idle_ns: 440795365367 ns186gitlab # [ 0.532148] Calibrating delay loop (skipped) preset value.. 6199.99 BogoMIPS (lpj=3099998)187gitlab # [ 0.534234] x86/cpu: User Mode Instruction Prevention (UMIP) activated188gitlab # [ 0.535345] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127189gitlab # [ 0.537143] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0190gitlab # [ 0.538152] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto191gitlab # [ 0.540146] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl192gitlab # [ 0.542144] Spectre V2 : Mitigation: Retpolines193gitlab # [ 0.544144] RETBleed: Mitigation: untrained return thunk194gitlab # [ 0.545144] Speculative Return Stack Overflow: Mitigation: SMT disabled195gitlab # [ 0.547144] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization196gitlab # [ 0.549144] Spectre V2 : Spectre v2 / SpectreRSB: Filling RSB on context switch and VMEXIT197gitlab # [ 0.551144] Spectre V2 : Enabling Speculation Barrier for firmware calls198gitlab # [ 0.552145] active return thunk: retbleed_return_thunk199gitlab # [ 0.554150] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier200gitlab # [ 0.555163] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'201gitlab # [ 0.557145] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'202gitlab # [ 0.559144] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'203gitlab # [ 0.560145] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256204gitlab # [ 0.561144] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'compacted' format.205gitlab # [ 0.588880] Freeing SMP alternatives memory: 44K206gitlab # [ 0.590148] pid_max: default: 32768 minimum: 301207gitlab # [ 0.591251] LSM: initializing lsm=capability,landlock,yama,bpf,ima208gitlab # [ 0.592265] landlock: Up and running.209gitlab # [ 0.593144] Yama: becoming mindful.210gitlab # [ 0.594356] LSM support for eBPF active211gitlab # [ 0.595412] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)212gitlab # [ 0.597316] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)213gitlab # [ 0.599912] smpboot: CPU0: AMD Ryzen 9 3900 12-Core Processor (family: 0x17, model: 0x71, stepping: 0x0)214gitlab # [ 0.600698] Performance Events: Fam17h+ core perfctr, AMD PMU driver.215gitlab # [ 0.601148] ... version: 0216gitlab # [ 0.602145] ... bit width: 48217gitlab # [ 0.603164] ... generic counters: 6218gitlab # [ 0.604145] ... generic bitmap: 000000000000003f219gitlab # [ 0.605146] ... fixed-purpose counters: 0220gitlab # [ 0.606145] ... fixed-purpose bitmap: 0000000000000000221gitlab # [ 0.607145] ... value mask: 0000ffffffffffff222gitlab # [ 0.608145] ... max period: 00007fffffffffff223gitlab # [ 0.609148] ... global_ctrl mask: 000000000000003f224gitlab # [ 0.610308] signal: max sigframe size: 1776225gitlab # [ 0.611298] rcu: Hierarchical SRCU implementation.226gitlab # [ 0.612153] rcu: Max phase no-delay instances is 400.227gitlab # [ 0.613411] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level228gitlab # [ 0.618382] smp: Bringing up secondary CPUs ...229gitlab # [ 0.619580] smpboot: x86: Booting SMP configuration:230gitlab # [ 0.620150] .... node #0, CPUs: #1 #2 #3231gitlab # [ 0.622199] smp: Brought up 1 node, 4 CPUs232gitlab # [ 0.624148] smpboot: Total of 4 processors activated (24799.98 BogoMIPS)233gitlab # [ 0.625673] Memory: 6013808K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 258088K reserved, 0K cma-reserved)234gitlab # [ 0.628541] devtmpfs: initialized235gitlab # [ 0.629453] x86/mm: Memory block size: 128MB236gitlab # [ 0.632324] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)237gitlab # [ 0.634263] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).238gitlab # [ 0.635324] pinctrl core: initialized pinctrl subsystem239gitlab # [ 0.636477] PM: RTC time: 10:31:11, date: 2026-09-01240gitlab # [ 0.640788] NET: Registered PF_NETLINK/PF_ROUTE protocol family241gitlab # [ 0.643774] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations242gitlab # [ 0.645318] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations243gitlab # [ 0.647651] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations244gitlab # [ 0.649186] audit: initializing netlink subsys (disabled)245gitlab # [ 0.650260] audit: type=2000 audit(1788258671.726:1): state=initialized audit_enabled=0 res=1246gitlab # [ 0.650482] thermal_sys: Registered thermal governor 'fair_share'247gitlab # [ 0.651147] thermal_sys: Registered thermal governor 'bang_bang'248gitlab # [ 0.652147] thermal_sys: Registered thermal governor 'step_wise'249gitlab # [ 0.653146] thermal_sys: Registered thermal governor 'user_space'250gitlab # [ 0.654147] thermal_sys: Registered thermal governor 'power_allocator'251gitlab # [ 0.655201] cpuidle: using governor menu252gitlab # [ 0.658490] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5253gitlab # [ 0.659526] PCI: Using configuration type 1 for base access254gitlab # [ 0.660150] PCI: Using configuration type 1 for extended access255gitlab # [ 0.661394] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.256gitlab # [ 0.663379] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages257gitlab # [ 0.665145] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page258gitlab # [ 0.667147] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages259gitlab # [ 0.669146] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page260gitlab # [ 0.674747] ACPI: Added _OSI(Module Device)261gitlab # [ 0.676147] ACPI: Added _OSI(Processor Device)262gitlab # [ 0.677146] ACPI: Added _OSI(Processor Aggregator Device)263gitlab # [ 0.680584] ACPI: 1 ACPI AML tables successfully acquired and loaded264gitlab # [ 0.684450] ACPI: Interpreter enabled265gitlab # [ 0.686243] ACPI: PM: (supports S0 S3 S4 S5)266gitlab # [ 0.687146] ACPI: Using IOAPIC for interrupt routing267gitlab # [ 0.689216] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug268gitlab # [ 0.691146] PCI: Using E820 reservations for host bridge windows269gitlab # [ 0.692393] ACPI: Enabled 2 GPEs in block 00 to 0F270gitlab # [ 0.697699] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])271gitlab # [ 0.699154] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]272gitlab # [ 0.701657] acpiphp: Slot [3] registered273gitlab # [ 0.703179] acpiphp: Slot [4] registered274gitlab # [ 0.704216] acpiphp: Slot [5] registered275gitlab # [ 0.705176] acpiphp: Slot [6] registered276gitlab # [ 0.706175] acpiphp: Slot [7] registered277gitlab # [ 0.707195] acpiphp: Slot [8] registered278gitlab # [ 0.708178] acpiphp: Slot [9] registered279gitlab # [ 0.710174] acpiphp: Slot [10] registered280gitlab # [ 0.711178] acpiphp: Slot [11] registered281gitlab # [ 0.712200] acpiphp: Slot [12] registered282gitlab # [ 0.713180] acpiphp: Slot [13] registered283gitlab # [ 0.714177] acpiphp: Slot [14] registered284gitlab # [ 0.715178] acpiphp: Slot [15] registered285gitlab # [ 0.716208] acpiphp: Slot [16] registered286gitlab # [ 0.718184] acpiphp: Slot [17] registered287gitlab # [ 0.719177] acpiphp: Slot [18] registered288gitlab # [ 0.720180] acpiphp: Slot [19] registered289gitlab # [ 0.721213] acpiphp: Slot [20] registered290gitlab # [ 0.722177] acpiphp: Slot [21] registered291gitlab # [ 0.723178] acpiphp: Slot [22] registered292gitlab # [ 0.724211] acpiphp: Slot [23] registered293gitlab # [ 0.726174] acpiphp: Slot [24] registered294gitlab # [ 0.727179] acpiphp: Slot [25] registered295gitlab # [ 0.728175] acpiphp: Slot [26] registered296gitlab # [ 0.729199] acpiphp: Slot [27] registered297gitlab # [ 0.730178] acpiphp: Slot [28] registered298gitlab # [ 0.731180] acpiphp: Slot [29] registered299gitlab # [ 0.733186] acpiphp: Slot [30] registered300gitlab # [ 0.734202] acpiphp: Slot [31] registered301gitlab # [ 0.735169] PCI host bridge to bus 0000:00302gitlab # [ 0.736178] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]303gitlab # [ 0.738147] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]304gitlab # [ 0.740147] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]305gitlab # [ 0.742148] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]306gitlab # [ 0.744150] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]307gitlab # [ 0.746149] pci_bus 0000:00: root bus resource [bus 00-ff]308gitlab # [ 0.747628] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint309gitlab # [ 0.750945] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint310gitlab # [ 0.754059] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint311gitlab # [ 0.757996] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f]312gitlab # [ 0.760205] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk313gitlab # [ 0.762148] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk314gitlab # [ 0.763150] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk315gitlab # [ 0.765150] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk316gitlab # [ 0.767566] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint317gitlab # [ 0.771105] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f]318gitlab # [ 0.773284] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint319gitlab # [ 0.776812] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI320gitlab # [ 0.778162] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB321gitlab # [ 0.780606] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint322gitlab # [ 0.792232] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]323gitlab # [ 0.794172] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]324gitlab # [ 0.796199] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]325gitlab # [ 0.797397] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]326gitlab # [ 0.801294] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint327gitlab # [ 0.806161] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]328gitlab # [ 0.808234] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]329gitlab # [ 0.810201] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]330gitlab # [ 0.812163] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]331gitlab # [ 0.816678] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint332gitlab # [ 0.823490] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f]333gitlab # [ 0.830168] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]334gitlab # [ 0.831199] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]335gitlab # [ 0.834140] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint336gitlab # [ 0.839162] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf]337gitlab # [ 0.840162] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]338gitlab # [ 0.842194] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]339gitlab # [ 0.845820] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint340gitlab # [ 0.850161] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df]341gitlab # [ 0.851160] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]342gitlab # [ 0.853197] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]343gitlab # [ 0.857283] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint344gitlab # [ 0.861181] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]345gitlab # [ 0.863163] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]346gitlab # [ 0.864200] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]347gitlab # [ 0.868571] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint348gitlab # [ 0.873160] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff]349gitlab # [ 0.874161] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]350gitlab # [ 0.876198] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]351gitlab # [ 0.879819] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint352gitlab # [ 0.886170] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff]353gitlab # [ 0.887160] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]354gitlab # [ 0.889198] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]355gitlab # [ 0.891177] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]356gitlab # [ 0.895336] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint357gitlab # [ 0.899179] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]358gitlab # [ 0.901207] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]359gitlab # [ 0.905172] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint360gitlab # [ 0.910182] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f]361gitlab # [ 0.912167] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]362gitlab # [ 0.913192] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]363gitlab # [ 0.916830] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint364gitlab # [ 0.921117] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f]365gitlab # [ 0.922158] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]366gitlab # [ 0.924194] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]367gitlab # [ 0.933319] ACPI: PCI: Interrupt link LNKA configured for IRQ 10368gitlab # [ 0.934359] ACPI: PCI: Interrupt link LNKB configured for IRQ 10369gitlab # [ 0.939131] ACPI: PCI: Interrupt link LNKC configured for IRQ 11370gitlab # [ 0.941327] ACPI: PCI: Interrupt link LNKD configured for IRQ 11371gitlab # [ 0.942247] ACPI: PCI: Interrupt link LNKS configured for IRQ 9372gitlab # [ 0.954266] iommu: Default domain type: Translated373gitlab # [ 0.956149] iommu: DMA domain TLB invalidation policy: lazy mode374gitlab # [ 0.963647] ACPI: bus type USB registered375gitlab # [ 0.964140] usbcore: registered new interface driver usbfs376gitlab # [ 0.964210] usbcore: registered new interface driver hub377gitlab # [ 0.965140] usbcore: registered new device driver usb378gitlab # [ 0.966902] NetLabel: Initializing379gitlab # [ 0.967140] NetLabel: domain hash size = 128380gitlab # [ 0.969151] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO381gitlab # [ 0.970598] NetLabel: unlabeled traffic allowed by default382gitlab # [ 0.971173] PCI: Using ACPI for IRQ routing383gitlab # [ 0.972947] pci 0000:00:02.0: vgaarb: setting as boot VGA device384gitlab # [ 0.973140] pci 0000:00:02.0: vgaarb: bridge control possible385gitlab # [ 0.973140] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none386gitlab # [ 0.976155] vgaarb: loaded387gitlab # [ 0.977517] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0388gitlab # [ 0.979153] hpet0: 3 comparators, 64-bit 100.000000 MHz counter389gitlab # [ 0.984516] clocksource: Switched to clocksource kvm-clock390gitlab # [ 0.988396] VFS: Disk quotas dquot_6.6.0391gitlab # [ 0.989743] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)392gitlab # [ 0.992649] pnp: PnP ACPI init393gitlab # [ 0.994450] pnp: PnP ACPI: found 6 devices394gitlab # [ 1.012100] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns395gitlab # [ 1.014622] clocksource: Switched to clocksource acpi_pm396gitlab # [ 1.016377] NET: Registered PF_INET protocol family397gitlab # [ 1.019256] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)398gitlab # [ 1.038490] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)399gitlab # [ 1.040726] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)400gitlab # [ 1.042888] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)401gitlab # [ 1.049504] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)402gitlab # [ 1.052758] TCP: Hash tables configured (established 65536 bind 65536)403gitlab # [ 1.055231] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)404gitlab # [ 1.057801] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)405gitlab # [ 1.060275] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)406gitlab # [ 1.062780] NET: Registered PF_UNIX/PF_LOCAL protocol family407gitlab # [ 1.064442] NET: Registered PF_XDP protocol family408gitlab # [ 1.065874] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]409gitlab # [ 1.067582] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]410gitlab # [ 1.069263] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]411gitlab # [ 1.071240] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]412gitlab # [ 1.073376] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]413gitlab # [ 1.075713] pci 0000:00:01.0: PIIX3: Enabling Passive Release414gitlab # [ 1.077575] pci 0000:00:00.0: Limiting direct PCI/PCI transfers415gitlab # [ 1.080686] ACPI: \_SB_.LNKD: Enabled at IRQ 11416gitlab # [ 1.083885] PCI: CLS 0 bytes, default 64417gitlab # [ 1.085163] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)418gitlab # [ 1.085588] Trying to unpack rootfs image as initramfs...419gitlab # [ 1.086964] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB)420gitlab # [ 1.087103] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x2caf45e6a57, max_idle_ns: 440795365367 ns421gitlab # [ 1.113121] Initialise system trusted keyrings422gitlab # [ 1.114823] workingset: timestamp_bits=40 max_order=21 bucket_order=0423gitlab # [ 1.131081] Key type asymmetric registered424gitlab # [ 1.132549] Asymmetric key parser 'x509' registered425gitlab # [ 1.134054] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)426gitlab # [ 1.136550] io scheduler mq-deadline registered427gitlab # [ 1.137964] io scheduler kyber registered428gitlab # [ 1.142741] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled429gitlab # [ 1.144748] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A430gitlab # [ 1.148514] Linux agpgart interface v0.103431gitlab # [ 1.149799] ACPI: bus type drm_connector registered432gitlab # [ 1.152843] usbcore: registered new interface driver usbserial_generic433gitlab # [ 1.154643] usbserial: USB Serial support registered for generic434gitlab # [ 1.156210] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled435gitlab # [ 1.158416] drop_monitor: Initializing network drop monitor service436gitlab # [ 1.160269] NET: Registered PF_INET6 protocol family437gitlab # [ 1.162864] Segment Routing with IPv6438gitlab # [ 1.163968] In-situ OAM (IOAM) with IPv6439gitlab # [ 1.167900] IPI shorthand broadcast: enabled440gitlab # [ 1.172440] sched_clock: Marking stable (941026098, 231293979)->(1267320339, -95000262)441gitlab # [ 1.175764] registered taskstats version 1442gitlab # [ 1.177655] Loading compiled-in X.509 certificates443gitlab # [ 1.185509] Demotion targets for Node 0: null444gitlab # [ 1.187349] Key type .fscrypt registered445gitlab # [ 1.188426] Key type fscrypt-provisioning registered446gitlab # [ 1.189864] ima: No TPM chip found, activating TPM-bypass!447gitlab # [ 1.191341] ima: Allocated hash algorithm: sha1448gitlab # [ 1.192644] ima: No architecture policies found449gitlab # [ 1.194239] PM: Magic number: 10:629:526450gitlab # [ 1.195451] faux: hash matches451gitlab # [ 1.197257] RAS: Correctable Errors collector initialized.452gitlab # [ 1.202949] clk: Disabling unused clocks453gitlab # [ 1.204099] PM: genpd: Disabling unused power domains454gitlab # [ 1.470949] Freeing initrd memory: 29392K455gitlab # [ 1.476950] Freeing unused decrypted memory: 2028K456gitlab # [ 1.481082] Freeing unused kernel image (initmem) memory: 3640K457gitlab # [ 1.482966] Write protecting the kernel read-only data: 32768k458gitlab # [ 1.485942] Freeing unused kernel image (text/rodata gap) memory: 1220K459gitlab # [ 1.488489] Freeing unused kernel image (rodata/data gap) memory: 760K460gitlab # [ 1.527217] x86/mm: Checked W+X mappings: passed, no W+X pages found.461gitlab # [ 1.528909] Run /init as init process462gitlab # [ 1.545162] systemd[1]: Inserted module 'autofs4'463gitlab # [ 1.580340] fuse: init (API version 7.45)464gitlab # [ 1.590238] ACPI: \_SB_.LNKC: Enabled at IRQ 10465gitlab # [ 1.599257] ACPI: \_SB_.LNKA: Enabled at IRQ 10466gitlab # [ 1.603252] ACPI: \_SB_.LNKB: Enabled at IRQ 11467gitlab # [ 1.648281] systemd[1]: Successfully made /usr/ read-only.468gitlab # [ 1.986707] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)469gitlab # [ 1.994353] systemd[1]: Detected virtualization kvm.470gitlab # [ 1.995754] systemd[1]: Detected architecture x86-64.471gitlab # [ 1.997081] systemd[1]: Running in initrd.472gitlab # [ 1.998852] systemd[1]: Initializing machine ID from random generator.473gitlab # [ 2.000552] systemd[1]: Hostname set to <gitlab>.474gitlab # [ 2.088933] systemd[1]: bpf-restrict-fs: LSM BPF program attached475gitlab # [ 2.128203] systemd[1]: Queued start job for default target Initrd Default Target.476gitlab # [ 2.144446] systemd[1]: Created slice Slice /system/modprobe.477gitlab # [ 2.146389] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.478gitlab # [ 2.148665] systemd[1]: Expecting device /dev/disk/by-label/nix-store...479gitlab # [ 2.150427] systemd[1]: Expecting device /dev/disk/by-label/nixos...480gitlab # [ 2.152346] systemd[1]: Reached target Path Units.481gitlab # [ 2.154258] systemd[1]: Reached target Slice Units.482gitlab # [ 2.156018] systemd[1]: Reached target Swaps.483gitlab # [ 2.157423] systemd[1]: Reached target Timer Units.484gitlab # [ 2.159136] systemd[1]: Listening on D-Bus System Message Bus Socket.485gitlab # [ 2.161234] systemd[1]: Listening on Journal Socket (/dev/log).486gitlab # [ 2.163022] systemd[1]: Listening on Journal Sockets.487gitlab # [ 2.164695] systemd[1]: Listening on udev Control Socket.488gitlab # [ 2.166436] systemd[1]: Listening on udev Kernel Socket.489gitlab # [ 2.167926] systemd[1]: Reached target Socket Units.490gitlab # [ 2.170780] systemd[1]: Starting Create List of Static Device Nodes...491gitlab # [ 2.173697] systemd[1]: Starting Load Kernel Module 9pnet_virtio...492gitlab # [ 2.182725] systemd[1]: Starting Load Kernel Module configfs...493gitlab # [ 2.186334] systemd[1]: Starting Journal Service...494gitlab # [ 2.192847] systemd[1]: Starting Load Kernel Modules...495gitlab # [ 2.194400] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os496gitlab # [ 2.200395] systemd[1]: Starting Coldplug All udev Devices...497gitlab # [ 2.209190] systemd[1]: Finished Create List of Static Device Nodes.498gitlab # [ 2.215977] systemd[1]: modprobe@configfs.service: Deactivated successfully.499gitlab # [ 2.218934] systemd[1]: Finished Load Kernel Module configfs.500gitlab # [ 2.221018] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config501gitlab # [ 2.227429] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...502gitlab # [ 2.227686] systemd-journald[92]: Collecting audit messages is disabled.503gitlab # [ 2.233661] netfs: FS-Cache loaded504gitlab # [ 2.235928] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.505gitlab # [ 2.244722] 9pnet: Installing 9P2000 support506gitlab # [ 2.249693] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev507gitlab # [ 2.255251] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.508gitlab # [ 2.258461] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.509gitlab # [ 2.261206] systemd[1]: Finished Load Kernel Module 9pnet_virtio.510gitlab # [ 2.264204] systemd[1]: Finished Load Kernel Modules.511gitlab # [ 2.267619] systemd[1]: Starting Apply Kernel Variables...512gitlab # [ 2.270648] systemd[1]: Starting Create Static Device Nodes in /dev...513gitlab # [ 2.286892] systemd[1]: Finished Apply Kernel Variables.514gitlab # [ 2.060691] systemd-modules-load[93]: Using 4[ 2.292556] systemd[1]: Finished Create Static Device Nodes in /dev.515gitlab # probe threads516gitlab # [ 2.294918] systemd[1]: Started Journal Service.517gitlab # [ 2.064337] systemd-modules-load[93]: Inserted module 'virtio_balloon'518gitlab # [ 2.066575] systemd-modules-load[93]: Inserted module 'virtio_gpu'519gitlab # [ 2.068451] systemd-modules-load[93]: Inserted module 'dm_mod'520gitlab # [ 2.070153] systemd[1]: Reached target Preparation for Local File Systems.521gitlab # [ 2.071973] systemd[1]: Reached target Local File Systems.522gitlab # [ 2.074652] systemd[1]: Starting Create System Files and Directories...523gitlab # [ 2.076568] systemd[1]: Starting Rule-based Manager for Device Events and Files...524gitlab # [ 2.082389] systemd[1]: Finished Create System Files and Directories.525gitlab # [ 2.083955] systemd[1]: Finished Coldplug All udev Devices.526gitlab # [ 2.092277] systemd-udevd[104]: Using default interface naming scheme 'v261'.527gitlab # [ 2.106056] systemd[1]: Started Rule-based Manager for Device Events and Files.528gitlab # [ 2.107724] systemd[1]: Reached target System Initialization.529gitlab # [ 2.109058] systemd[1]: Reached target Basic System.530gitlab # [ 2.466929] virtio_blk virtio4: 4/0/0 default/read/poll queues531gitlab # [ 2.474177] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)532gitlab # [ 2.479599] virtio_blk virtio5: 4/0/0 default/read/poll queues533gitlab # [ 2.483360] SCSI subsystem initialized534gitlab # [ 2.491572] virtio_blk virtio5: [vdb] 10455032 512-byte logical blocks (5.35 GB/4.98 GiB)535gitlab # [ 2.498093] uhci_hcd 0000:00:01.2: UHCI Host Controller536gitlab # [ 2.501063] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1537gitlab # [ 2.503123] uhci_hcd 0000:00:01.2: detected 2 ports538gitlab # [ 2.512569] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140539gitlab # [ 2.515380] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18540gitlab # [ 2.517202] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1541gitlab # [ 2.518865] usb usb1: Product: UHCI Host Controller542gitlab # [ 2.520041] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd543gitlab # [ 2.521684] usb usb1: SerialNumber: 0000:00:01.2544gitlab # [ 2.529415] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12545gitlab # [ 2.536597] serio: i8042 KBD port at 0x60,0x64 irq 1546gitlab # [ 2.537250] hub 1-0:1.0: USB hub found547gitlab # [ 2.537749] serio: i8042 AUX port at 0x60,0x64 irq 12548gitlab # [ 2.539846] hub 1-0:1.0: 2 ports detected549gitlab # [ 2.556964] scsi host0: ata_piix550gitlab # [ 2.558334] scsi host1: ata_piix551gitlab # [ 2.559141] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0552gitlab # [ 2.560776] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0553gitlab # [ 2.336321] (udev-worker)[120]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.554gitlab # [ 2.339579] (udev-worker)[120]: Network interface NamePolicy= disabled on kernel command line.555gitlab # [ 2.342811] (udev-worker)[130]: Network interface NamePolicy= disabled on kernel command line.556gitlab # [ 2.351774] systemd[1]: Starting Virtual Console Setup...557gitlab # [ 2.364543] systemd[1]: Found device /dev/disk/by-label/nix-store.558gitlab # [ 2.369153] systemd[1]: Found device /dev/disk/by-label/nixos.559gitlab # [ 2.370623] systemd[1]: Reached target Initrd Root Device.560gitlab # [ 2.371841] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...561gitlab # [ 2.376803] systemd-vconsole-setup[150]: Configuration of first virtual console was skipped, ignoring remaining ones.562gitlab # [ 2.382169] systemd[1]: Finished Virtual Console Setup.563gitlab # [ 2.397967] systemd-fsck[155]: nixos: clean, 12/65536 files, 13019/262144 blocks564gitlab # [ 2.402464] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.565gitlab # [ 2.728622] ata2: found unknown device (class 0)566gitlab # [ 2.730858] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100567gitlab # [ 2.734345] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5568gitlab # [ 2.759606] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0569gitlab # [ 2.776368] usb 1-1: new full-speed USB device number 2 using uhci_hcd570gitlab # [ 2.788518] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray571gitlab # [ 2.819999] cdrom: Uniform CD-ROM driver Revision: 3.20572gitlab # [ 2.943663] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00573gitlab # [ 2.945500] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10574gitlab # [ 2.947188] usb 1-1: Product: QEMU USB Tablet575gitlab # [ 2.948263] usb 1-1: Manufacturer: QEMU576gitlab # [ 2.949230] usb 1-1: SerialNumber: 28754-0000:00:01.2-1577gitlab # [ 2.966261] hid: raw HID events driver (C) Jiri Kosina578gitlab # [ 2.974849] usbcore: registered new interface driver usbhid579gitlab # [ 2.976429] usbhid: USB HID core driver580gitlab # [ 2.981513] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2581gitlab # [ 2.984910] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0582gitlab # [ 2.976625] systemd[1]: Mounting /sysroot...583gitlab # [ 3.350620] EXT4-fs (vda): mounted filesystem 91db64c9-2ec4-48f0-9355-e89f85a25746 r/w with ordered data mode. Quota mode: none.584gitlab # [ 3.123776] systemd[1]: Mounted /sysroot.585gitlab # [ 3.125381] systemd[1]: Reached target Initrd Root File System.586gitlab # [ 3.128422] systemd[1]: Mounting /sysroot/nix/.ro-store...587gitlab # [ 3.130228] systemd[1]: Mounting /sysroot/nix/.rw-store...588gitlab # [ 3.137134] systemd[1]: Mounting /sysroot/run...589gitlab # [ 3.141212] systemd[1]: Mounting /sysroot/tmp/shared...590gitlab # [ 3.143640] systemd[1]: Mounting /sysroot/tmp/xchg...591gitlab # [ 3.146442] systemd[1]: Starting Mountpoints Configured in the Real Root...592gitlab # [ 3.381903] erofs (device vdb): mounted with root inode @ nid 128.593gitlab # [ 3.153503] systemd[1]: Mounted /sysroot/nix/.rw-store.594gitlab # [ 3.155121] systemd-sysroot-fstab-check[176]: /sysroot should be mounted in the initrd, will request daemon-reload.595gitlab # [ 3.157530] systemd[1]: Mounted /sysroot/run.596gitlab # [ 3.158711] systemd[1]: Mounted /sysroot/nix/.ro-store.597gitlab # [ 3.160272] s[ 3.391259] 9p: Installing v9fs 9p2000 file system support598gitlab # ystemd[1]: Starting rw-sysroot-nix-store.service...599gitlab # [ 3.163185] systemd[1]: Reload requested from client PID 176 ('systemd-sysroot') (unit initrd-parse-etc.service)...600gitlab # [ 3.166072] systemd[1]: Reloading...601gitlab # [ 3.236125] systemd[1]: Reloading finished in 72 ms.602gitlab # [ 3.258881] systemd-sysroot-fstab-check[176]: Requesting initrd-fs.target/start/replace...603gitlab # [ 3.261885] systemd[1]: Mounted /sysroot/tmp/shared.604gitlab # [ 3.263717] systemd[1]: Mounted /sysroot/tmp/xchg.605gitlab # [ 3.265400] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.606gitlab # [ 3.267297] systemd[1]: Finished rw-sysroot-nix-store.service.607gitlab # [ 3.268700] systemd-sysroot-fstab-check[176]: Requesting swap.target/start/replace...608gitlab # [ 3.270587] systemd[1]: Starting rw-sysroot-nix-store.service...609gitlab # [ 3.272286] systemd[1]: initrd-parse-etc.service: Deactivated successfully.610gitlab # [ 3.273952] systemd[1]: Finished Mountpoints Configured in the Real Root.611gitlab # [ 3.275549] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.612gitlab # [ 3.286636] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.613gitlab # [ 3.288613] systemd[1]: Finished rw-sysroot-nix-store.service.614gitlab # [ 3.973777] systemd[1]: Mounting /sysroot/nix/store...615gitlab # [ 4.002046] systemd[1]: Mounted /sysroot/nix/store.616gitlab # [ 4.003370] systemd[1]: Reached target Initrd File Systems.617gitlab # [ 4.004519] systemd[1]: Starting Find NixOS closure...618gitlab # [ 4.005959] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...619gitlab # [ 4.021268] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.620gitlab # [ 4.023282] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.621gitlab # [ 4.034518] systemd[1]: Finished Find NixOS closure.622gitlab # [ 4.036529] systemd[1]: Reached target Initrd Default Target.623gitlab # [ 4.037916] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...624gitlab # [ 4.056654] systemd[1]: Stopped target Initrd Default Target.625gitlab # [ 4.058330] systemd[1]: Stopped target Basic System.626gitlab # [ 4.059857] systemd[1]: Stopped target Initrd Root Device.627gitlab # [ 4.061218] systemd[1]: Stopped target Path Units.628gitlab # [ 4.062440] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.629gitlab # [ 4.064140] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.630gitlab # [ 4.066062] systemd[1]: Stopped target Slice Units.631gitlab # [ 4.067424] systemd[1]: Stopped target Socket Units.632gitlab # [ 4.068662] systemd[1]: Stopped target System Initialization.633gitlab # [ 4.070177] systemd[1]: Stopped target Swaps.634gitlab # [ 4.071478] systemd[1]: Stopped target Timer Units.635gitlab # [ 4.072826] systemd[1]: dbus.socket: Deactivated successfully.636gitlab # [ 4.074499] systemd[1]: Closed D-Bus System Message Bus Socket.637gitlab # [ 4.075869] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.638gitlab # [ 4.077615] systemd[1]: Stopped Find NixOS closure.639gitlab # [ 4.078982] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio640gitlab # [ 4.081440] systemd[1]: Starting rw-sysroot-nix-store.service...641gitlab # [ 4.082753] systemd[1]: systemd-sysctl.service: Deactivated successfully.642gitlab # [ 4.084165] systemd[1]: Stopped Apply Kernel Variables.643gitlab # [ 4.086197] systemd[1]: systemd-modules-load.service: Deactivated successfully.644gitlab # [ 4.087788] systemd[1]: Stopped Load Kernel Modules.645gitlab # [ 4.089298] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.646gitlab # [ 4.091281] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.647gitlab # [ 4.093182] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.648gitlab # [ 4.094916] systemd[1]: Stopped Create System Files and Directories.649gitlab # [ 4.096336] systemd[1]: Stopped target Local File Systems.650gitlab # [ 4.097544] systemd[1]: Stopped target Preparation for Local File Systems.651gitlab # [ 4.099089] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.652gitlab # [ 4.102124] systemd[1]: Stopped Coldplug All udev Devices.653gitlab # [ 4.103675] systemd[1]: Stopping Rule-based Manager for Device Events and Files...654gitlab # [ 4.105430] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.655gitlab # [ 4.106972] systemd[1]: Stopped Virtual Console Setup.656gitlab # [ 4.108490] systemd[1]: initrd-cleanup.service: Deactivated successfully.657gitlab # [ 4.110203] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.658gitlab # [ 4.111929] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.659gitlab # [ 4.113721] systemd[1]: Finished rw-sysroot-nix-store.service.660gitlab # [ 4.115218] systemd[1]: systemd-udevd.service: Deactivated successfully.661gitlab # [ 4.116675] systemd[1]: Stopped Rule-based Manager for Device Events and Files.662gitlab # [ 4.118454] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.663gitlab # [ 4.120091] systemd[1]: Closed udev Control Socket.664gitlab # [ 4.121200] systemd[1]: Starting Cleanup udev Database...665gitlab # [ 4.122389] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.666gitlab # [ 4.124295] systemd[1]: Stopped Create Static Device Nodes in /dev.667gitlab # [ 4.125804] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.668gitlab # [ 4.127617] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.669gitlab # [ 4.129400] systemd[1]: kmod-static-nodes.service: Deactivated successfully.670gitlab # [ 4.130902] systemd[1]: Stopped Create List of Static Device Nodes.671gitlab # [ 4.133302] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.672gitlab # [ 4.135529] systemd[1]: Finished Cleanup udev Database.673gitlab # [ 4.137303] systemd[1]: Reached target Switch Root.674gitlab # [ 4.138564] systemd[1]: Starting NixOS Activation...675gitlab # [ 4.203754] initrd-nixos-activation-start[224]: booting system configuration /nix/store/2lzp8hvaxzcmd3b3cpwanxzjxxlviix8-nixos-system-gitlab-test676gitlab # [ 4.234468] initrd-nixos-activation-start[224]: running activation script...677gitlab # [ 4.563138] initrd-nixos-activation-start[247]: setting up /etc...678gitlab # [ 4.666819] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.679gitlab # [ 4.668807] systemd[1]: Finished NixOS Activation.680gitlab # [ 4.670837] systemd[1]: Starting Switch Root...681gitlab # [ 4.696531] systemd[1]: Switching root.682gitlab # [ 4.982486] systemd-journald[92]: Received SIGTERM from PID 1 (systemd).683gitlab # [ 5.075456] NET: Registered PF_VSOCK protocol family684gitlab # [ 5.443994] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)685gitlab # [ 5.450032] systemd[1]: Detected virtualization kvm.686gitlab # [ 5.451120] systemd[1]: Detected architecture x86-64.687gitlab # [ 5.452399] systemd[1]: Detected first boot.688gitlab # [ 5.453872] systemd[1]: Initializing machine ID from random generator.689gitlab # [ 5.562860] systemd[1]: bpf-restrict-fs: LSM BPF program attached690gitlab # [ 5.625096] systemd[1]: Applying preset policy.691gitlab # [ 5.746796] systemd[1]: Populated /etc with preset unit settings.692gitlab # [ 5.908402] systemd[1]: initrd-switch-root.service: Deactivated successfully.693gitlab # [ 5.911018] systemd[1]: Stopped initrd-switch-root.service.694gitlab # [ 5.914749] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.695gitlab # [ 5.918323] systemd[1]: Created slice Slice /system/getty.696gitlab # [ 5.920484] systemd[1]: Created slice GitLab DevOps Platform Slice.697gitlab # [ 5.922739] systemd[1]: Created slice User and Session Slice.698gitlab # [ 5.924287] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.699gitlab # [ 5.926203] systemd[1]: Started Forward Password Requests to Wall Directory Watch.700gitlab # [ 5.927997] systemd[1]: Expecting device /dev/hvc0...701gitlab # [ 5.929130] systemd[1]: Expecting device /dev/ttyS0...702gitlab # [ 5.930577] systemd[1]: Reached target Local Encrypted Volumes.703gitlab # [ 5.932178] systemd[1]: Stopped target initrd-fs.target.704gitlab # [ 5.933511] systemd[1]: Stopped target initrd-root-fs.target.705gitlab # [ 5.934861] systemd[1]: Stopped target initrd-switch-root.target.706gitlab # [ 5.936373] systemd[1]: Reached target Virtual Machines and Containers.707gitlab # [ 5.938904] systemd[1]: Reached target Path Units.708gitlab # [ 5.940310] systemd[1]: Reached target Remote File Systems.709gitlab # [ 5.941617] systemd[1]: Reached target Slice Units.710gitlab # [ 5.942772] systemd[1]: Reached target Swaps.711gitlab # [ 5.945002] systemd[1]: Listening on Query the User Interactively for a Password.712gitlab # [ 5.947687] systemd[1]: Listening on Process Core Dump Socket.713gitlab # [ 5.949712] systemd[1]: Listening on Credential Encryption/Decryption.714gitlab # [ 5.951948] systemd[1]: Listening on Factory Reset Management.715gitlab # [ 5.953391] systemd[1]: Listening on Hostname Service Socket.716gitlab # [ 5.956588] systemd[1]: Starting Journal Log Access Socket...717gitlab # [ 5.958664] systemd[1]: Listening on Journal Audit Socket.718gitlab # [ 5.961197] systemd[1]: Listening on Console Output Muting Service Socket.719gitlab # [ 5.963045] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.720gitlab # [ 5.964908] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os721gitlab # [ 5.967005] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki722gitlab # [ 5.970988] systemd[1]: Listening on Disk Repartitioning Service Socket.723gitlab # [ 5.972737] systemd[1]: Listening on udev Control Socket.724gitlab # [ 5.974196] systemd[1]: Listening on udev Varlink Socket.725gitlab # [ 5.977728] systemd[1]: Mounting Huge Pages File System...726gitlab # [ 5.980635] systemd[1]: Mounting POSIX Message Queue File System...727gitlab # [ 5.983408] systemd[1]: Mounting Kernel Debug File System...728gitlab # [ 5.986400] systemd[1]: Mounting Kernel Trace File System...729gitlab # [ 5.989698] systemd[1]: Starting Create List of Static Device Nodes...730gitlab # [ 5.991514] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio731gitlab # [ 5.998218] systemd[1]: Starting Load Kernel Module configfs...732gitlab # [ 5.999846] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm733gitlab # [ 6.002585] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore734gitlab # [ 6.005217] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse735gitlab # [ 6.008808] systemd[1]: Mounting FUSE Control File System...736gitlab # [ 6.010523] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67737gitlab # [ 6.016213] systemd[1]: Starting Journal Service...738gitlab # [ 6.024033] systemd[1]: Starting Load Kernel Modules...739gitlab # [ 6.027010] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...740gitlab # [ 6.032436] systemd[1]: Starting Remount Root and Kernel File Systems...741gitlab # [ 6.034119] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os742gitlab # [ 6.038495] systemd[1]: Starting Coldplug All udev Devices...743gitlab # [ 6.039127] systemd-journald[318]: Collecting audit messages is enabled.744gitlab # [ 6.042927] systemd[1]: Listening on Journal Log Access Socket.745gitlab # [ 6.046807] systemd[1]: Mounted Huge Pages File System.746gitlab # [ 6.048213] systemd[1]: Mounted POSIX Message Queue File System.747gitlab # [ 6.049781] systemd[1]: Mounted Kernel Debug File System.748gitlab # [ 6.050471] loop: module loaded749gitlab # [ 6.052329] systemd[1]: Mounted Kernel Trace File System.750gitlab # [ 6.054179] EXT4-fs (vda): re-mounted 91db64c9-2ec4-48f0-9355-e89f85a25746.751gitlab # [ 6.058484] systemd[1]: Finished Create List of Static Device Nodes.752gitlab # [ 6.060587] systemd[1]: modprobe@configfs.service: Deactivated successfully.753gitlab # [ 6.062930] systemd[1]: Finished Load Kernel Module configfs.754gitlab # [ 6.064799] systemd[1]: Mounted FUSE Control File System.755gitlab # [ 6.066871] systemd[1]: Finished Load Kernel Modules.756gitlab # [ 5.837282] s[ 6.068497] systemd[1]: Started Journal Service.757gitlab # ystemd[1]: Queued start job for default target Multi-User System.758gitlab # [ 5.840789] systemd[1]: Unnecessary job was removed for /dev/vdb.759gitlab # [ 5.842699] systemd[1]: systemd-journald.service: Deactivated successfully.760gitlab # [ 5.844652] systemd-modules-load[320]: Using 3 probe threads761gitlab # [ 5.846459] systemd-modules-load[320]: Inserted module 'loop'762gitlab # [ 5.850196] systemd-modules-load[320]: Inserted module 'tls'763gitlab # [ 5.851838] systemd[1]: Finished Remount Root and Kernel File Systems.764gitlab # [ 5.854115] systemd[1]: Listening on Disk Image Download Service Socket.765gitlab # [ 5.856247] systemd[1]: Mounting Kernel Configuration File System...766gitlab # [ 5.858325] systemd[1]: Starting Firewall...767gitlab # [ 5.859778] systemd[1]: Starting Flush Journal to Persistent Storage...768gitlab # [ 5.861799] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore769gitlab # [ 5.867482] systemd-oomd[321]: No swap; memory pressure usage will be degraded770gitlab # [ 5.870208] systemd[1]: Starting Load/Save OS Random Seed...771gitlab # [ 5.872236] systemd[1]: Starting Apply Kernel Variables...772gitlab # [ 5.873987] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...773gitlab # [ 5.876143] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os774gitlab # [ 5.878565] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.775gitlab # [ 5.881670] systemd[1]: Mounted Kernel Configuration File System.776gitlab # [ 6.119226] systemd-journald[318]: Received client request to flush runtime journal.777gitlab # [ 5.954751] systemd[1]: Finished Load/Save OS Random Seed.778gitlab # [ 5.957076] systemd[1]: Reached target First Boot Complete.779gitlab # [ 5.959118] systemd[1]: Finished Apply Kernel Variables.780gitlab # [ 5.960631] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.781gitlab # [ 5.962578] systemd[1]: Starting Create Static Device Nodes in /dev...782gitlab # [ 5.964503] systemd[1]: Finished Coldplug All udev Devices.783gitlab # [ 5.965894] systemd[1]: Finished Create Static Device Nodes in /dev.784gitlab # [ 5.967392] systemd[1]: Reached target Preparation for Local File Systems.785gitlab # [ 5.969438] systemd[1]: Starting Rule-based Manager for Device Events and Files...786gitlab # [ 5.971385] systemd[1]: Finished Flush Journal to Persistent Storage.787gitlab # [ 5.974361] systemd-udevd[365]: Using default interface naming scheme 'v261'.788gitlab # [ 6.016821] systemd[1]: Started Rule-based Manager for Device Events and Files.789gitlab # [ 6.120981] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse790gitlab # [ 6.146805] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.791gitlab # [ 6.155124] systemd[1]: Finished Firewall.792gitlab # [ 6.178498] (udev-worker)[429]: Network interface NamePolicy= disabled on kernel command line.793gitlab # [ 6.181422] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.794gitlab # [ 6.186609] (udev-worker)[425]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.795gitlab # [ 6.190205] (udev-worker)[425]: Network interface NamePolicy= disabled on kernel command line.796gitlab # [ 6.213394] systemd[1]: Condition check resulted in Virtio network device being skipped.797gitlab # [ 6.217987] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore798gitlab # [ 6.220554] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67799gitlab # [ 6.224129] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore800gitlab # [ 6.226703] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os801gitlab # [ 6.228723] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os802gitlab # [ 6.483333] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0803gitlab # [ 6.486145] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console804gitlab # [ 6.486309] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD805gitlab # [ 6.490506] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3806gitlab # [ 6.497735] Console: switching to colour dummy device 80x25807gitlab # [ 6.500431] [drm] Found bochs VGA, ID 0xb0c5.808gitlab # [ 6.501178] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.809gitlab # [ 6.502757] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic810gitlab # [ 6.503905] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0811gitlab # [ 6.510113] ACPI: button: Power Button [PWRF]812gitlab # [ 6.514127] mousedev: PS/2 mouse device common for all mice813gitlab # [ 6.531768] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5814gitlab # [ 6.535142] Console: switching to colour frame buffer device 160x50815gitlab # [ 6.541800] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device816gitlab # [ 6.546649] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4817gitlab # [ 6.548657] parport_pc 00:03: reported by Plug and Play ACPI818gitlab # [ 6.550667] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)]819gitlab # [ 6.552197] Floppy drive(s): fd0 is 2.88M AMI BIOS820gitlab # [ 6.559666] rtc_cmos 00:05: RTC can wake from S4821gitlab # [ 6.565704] FDC 0 is a S82078B822gitlab # [ 6.570232] rtc_cmos 00:05: registered as rtc0823gitlab # [ 6.573012] rtc_cmos 00:05: setting system clock to 2026-09-01T10:31:17 UTC (1788258677)824gitlab # [ 6.574994] systemd-journald[318]: Time jumped backwards, rotating.825gitlab # [ 6.579732] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs826gitlab # [ 6.587296] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6827gitlab # [ 6.429606] systemd[1]: Starting Virtual Console Setup...828gitlab # [ 6.442873] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.829gitlab # [ 6.444743] systemd[1]: Stopped Virtual Console Setup.830gitlab # [ 6.452460] systemd[1]: Starting Virtual Console Setup...831gitlab # [ 6.707340] erofs (device vdb): failed to read inode meta block (nid: 109543273): -4832gitlab # [ 6.479140] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.833gitlab # [ 6.481298] systemd[1]: Stopped Virtual Console Setup.834gitlab # [ 6.716341] ppdev: user-space parallel port driver835gitlab # [ 6.486875] systemd[1]: Starting Virtual Console Setup...836gitlab # [ 6.845440] kvm_amd: TSC scaling supported837gitlab # [ 6.846914] kvm_amd: Nested Virtualization enabled838gitlab # [ 6.847907] kvm_amd: Nested Paging enabled839gitlab # [ 6.848753] kvm_amd: LBR virtualization supported840gitlab # [ 6.849727] kvm_amd: Virtual VMLOAD VMSAVE supported841gitlab # [ 6.850786] kvm_amd: Virtual GIF supported842gitlab # [ 6.636178] systemd-vconsole-setup[493]: Configuration of first virtual console was skipped, ignoring remaining ones.843gitlab # [ 6.640761] systemd[1]: Finished Virtual Console Setup.844gitlab # [ 6.903154] EDAC MC: Ver: 3.0.0845gitlab # [ 6.680144] systemd[1]: Mounting /run/wrappers...846gitlab # [ 6.699112] systemd[1]: Mounted /run/wrappers.847gitlab # [ 6.700710] systemd[1]: Reached target Local File Systems.848gitlab # [ 6.702613] systemd[1]: Listening on Boot Loader Control Service Socket.849gitlab # [ 6.705273] systemd[1]: Starting register-nix-paths.service...850gitlab # [ 6.707705] systemd[1]: Starting Create SUID/SGID Wrappers...851gitlab # [ 6.710382] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.852gitlab # [ 6.725229] systemd[1]: Starting Save Transient machine-id to Disk...853gitlab # [ 6.727298] systemd[1]: Starting Create System Files and Directories...854gitlab # [ 6.749559] systemd[1]: Finished Create System Files and Directories.855gitlab # [ 6.754732] systemd[1]: Starting Rebuild Journal Catalog...856gitlab # [ 6.756661] systemd[1]: Starting Record System Boot/Shutdown in UTMP...857gitlab # [ 6.762169] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.858gitlab # [ 6.766292] systemd[1]: Finished Save Transient machine-id to Disk.859gitlab # [ 6.779837] systemd[1]: Finished Record System Boot/Shutdown in UTMP.860gitlab # [ 6.797885] systemd[1]: Finished Rebuild Journal Catalog.861gitlab # [ 6.801373] systemd[1]: Starting Update is Completed...862gitlab # [ 6.819359] systemd[1]: Finished Update is Completed.863gitlab # [ 6.862112] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.864gitlab # [ 6.863964] systemd[1]: Finished Create SUID/SGID Wrappers.865gitlab # [ 7.179416] systemd[1]: Finished register-nix-paths.service.866gitlab # [ 7.181378] systemd[1]: Reached target System Initialization.867gitlab # [ 7.183278] systemd[1]: Started Discard unused filesystem blocks once a week.868gitlab # [ 7.184969] systemd[1]: Started logrotate.timer.869gitlab # [ 7.186203] systemd[1]: Started Daily Cleanup of Temporary Directories.870gitlab # [ 7.188060] systemd[1]: Reached target Timer Units.871gitlab # [ 7.189375] systemd[1]: Listening on D-Bus System Message Bus Socket.872gitlab # [ 7.190820] systemd[1]: Listening on Nix Daemon Socket.873gitlab # [ 7.193868] systemd[1]: Starting nixbot CI service socket...874gitlab # [ 7.195792] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.875gitlab # [ 7.197820] systemd[1]: Listening on nixbot CI service socket.876gitlab # [ 7.199465] systemd[1]: Reached target Socket Units.877gitlab # [ 7.241242] systemd[1]: Starting D-Bus System Message Bus...878gitlab # [ 7.431865] dbus-broker-launch[576]: Looking up NSS user entry for 'systemd-timesync'...879gitlab # [ 7.433969] dbus-broker-launch[576]: NSS returned no entry for 'systemd-timesync'880gitlab # [ 7.435689] dbus-broker-launch[576]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"881gitlab # [ 7.441401] systemd[1]: Started D-Bus System Message Bus.882gitlab # [ 7.460883] dbus-broker-launch[576]: Ready883gitlab # [ 7.462515] systemd[1]: Reached target Basic System.884gitlab # [ 7.466392] systemd[1]: Started backdoor.service.885gitlab # [ 7.468073] systemd[1]: Starting gitlab-config.service...886gitlab # [ 7.470426] systemd[1]: Starting Import lastlog data into lastlog2 database...887gitlab # [ 7.473374] systemd[1]: Starting Logrotate configuration check...888gitlab # [ 7.476887] systemd[1]: Starting Name Service Cache Daemon (nsncd)...889gitlab # [ 7.487179] systemd[1]: Starting Post-Boot Actions...890gitlab # [ 7.494695] systemd[1]: Started Reset console on configuration changes.891gitlab # connecting to host...892gitlab # [ 7.496261] logrotate[581]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.893gitlab # [ 7.498678] logrotate[581]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf894gitlab # [ 7.500465] logrotate[581]: note: 'monthly' overrides previously specified 'weekly'895gitlab # [ 7.502494] logrotate[581]: Reading state from file: /var/lib/logrotate.status896gitlab # [ 7.503893] logrotate[581]: state file /var/lib/logrotate.status does not exist897gitlab # [ 7.504657] logrotate[581]: Allocating hash table for state file, size 64 entries898gitlab # [ 7.505727] logrotate[581]: Handling 4 logs899gitlab # [ 7.506645] logrotate[581]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed900gitlab # [ 7.507719] logrotate[581]: considering log /var/log/btmp901gitlab # [ 7.508623] logrotate[581]: Creating new state902gitlab # [ 7.509758] logrotate[581]: Now: 2026-09-01 10:31903gitlab # [ 7.510787] logrotate[581]: Last rotated at 2026-09-01 10:00904gitlab: Guest shell says: b'Spawning backdoor root shell...\n'905gitlab # [ 7.511799] logrotate[581]: log does not need rotating (log has already been rotated)906gitlab # [ 7.512844] logrotate[581]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed907gitlab # [ 7.514046] logrotate[581]: considering log /var/log/wtmp908gitlab # [ 7.514735] logrotate[581]: Creating new state909gitlab # [ 7.516208] logrotate[581]: Now: 2026-09-01 10:31910gitlab # [ 7.517372] logrotate[581]: Last rotated at 2026-09-01 10:00911gitlab # [ 7.518047] logrotate[581]: log does not need rotating (log has already been rotated)912gitlab # [ 7.518802] logrotate[581]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed913gitlab # [ 7.519794] logrotate[581]: switching euid from 0 to 165 and egid from 0 to 165 (pid 581)914gitlab # [ 7.520799] logrotate[581]: considering log /var/gitlab/state/log/*.log915gitlab # [ 7.521799] logrotate[581]: log /var/gitlab/state/log/*.log does not exist -- skipping916gitlab # [ 7.522752] logrotate[581]: Creating new state917gitlab # [ 7.523886] logrotate[581]: switching euid from 165 to 0 and egid from 165 to 0 (pid 581)918gitlab # [ 7.524991] logrotate[581]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed919gitlab: connected to guest root shell920gitlab: (connecting took 8.45 seconds)921gitlab: (finished: waiting for the VM to finish booting, in 8.45 seconds)922gitlab # [ 7.525793] logrotate[581]: switching euid from 0 to 60 and egid from 0 to 60 (pid 581)923gitlab # [ 7.526810] logrotate[581]: considering log /var/log/nginx/*.log924gitlab # [ 7.527845] logrotate[581]: log /var/log/nginx/*.log does not exist -- skipping925gitlab # [ 7.528639] logrotate[581]: Creating new state926gitlab # [ 7.530236] logrotate[581]: not running postrotate script, since no logs were rotated927gitlab # [ 7.530904] logrotate[581]: switching euid from 60 to 0 and egid from 60 to 0 (pid 581)928gitlab # [ 7.551230] systemd[1]: Starting resolvconf update...929gitlab # [ 7.563484] systemd[1]: Finished Logrotate configuration check.930gitlab # [ 7.565390] nsncd[582]: Sep 01 10:31:18.721 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"931gitlab # [ 7.569360] systemd[1]: Started Name Service Cache Daemon (nsncd).932gitlab # [ 7.570653] systemd[1]: Finished Post-Boot Actions.933gitlab # [ 7.578493] systemd[1]: Finished Import lastlog data into lastlog2 database.934gitlab # [ 7.586917] systemd[1]: Reached target Host and Network Name Lookups.935gitlab # [ 7.587809] systemd[1]: Reached target User and Group Name Lookups.936gitlab # [ 7.595960] systemd[1]: Starting User Login Management...937gitlab # [ 7.624516] systemd[1]: Stopped target Host and Network Name Lookups.938gitlab # [ 7.626326] systemd[1]: Stopping Host and Network Name Lookups...939gitlab # [ 7.627194] systemd[1]: Stopped target User and Group Name Lookups.940gitlab # [ 7.629347] systemd[1]: Stopping User and Group Name Lookups...941gitlab # [ 7.629970] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...942gitlab # [ 7.633197] systemd[1]: nscd.service: Deactivated successfully.943gitlab # [ 7.635448] systemd[1]: Stopped Name Service Cache Daemon (nsncd).944gitlab # [ 7.637781] systemd[1]: nscd.service: Consumed 67ms CPU time over 156ms wall clock time, 7.3M memory peak, 5.5M read from disk.945gitlab # [ 7.641971] systemd[1]: Starting Name Service Cache Daemon (nsncd)...946gitlab # [ 7.647845] systemd-logind[628]: New seat seat0.947gitlab # [ 7.656979] systemd-logind[628]: Watching system buttons on /dev/input/event2 (Power Button)948gitlab # [ 7.661167] systemd-logind[628]: Watching system buttons on /dev/input/event5 (QEMU Virtio Keyboard)949gitlab # [ 7.664123] systemd-logind[628]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)950gitlab # [ 7.665215] systemd[1]: Started User Login Management.951gitlab # [ 7.667045] nsncd[673]: Sep 01 10:31:18.822 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"952gitlab # [ 7.667814] systemd[1]: Starting linger-users.service...953gitlab # [ 7.670706] systemd[1]: Started Name Service Cache Daemon (nsncd).954gitlab # [ 7.671508] systemd[1]: Reached target Host and Network Name Lookups.955gitlab # [ 7.672453] systemd[1]: Reached target User and Group Name Lookups.956gitlab # [ 7.679486] systemd[1]: Finished resolvconf update.957gitlab # [ 7.682113] systemd[1]: Reached target Preparation for Network.958gitlab # [ 7.687366] systemd[1]: Starting DHCP Client...959gitlab # [ 7.689868] systemd[1]: Starting Address configuration of eth1...960gitlab # [ 7.693125] systemd[1]: Starting Extra networking commands....961gitlab # [ 7.696163] systemd[1]: linger-users.service: Deactivated successfully.962gitlab # [ 7.703501] systemd[1]: Finished linger-users.service.963gitlab # [ 7.725864] network-addresses-eth1-start[698]: adding address 192.168.1.1/24... done964gitlab # [ 7.731586] network-addresses-eth1-start[698]: adding address 2001:db8:1::1/64... done965gitlab # [ 7.739855] systemd[1]: Finished Address configuration of eth1.966gitlab # [ 7.774832] systemd[1]: Finished Extra networking commands..967gitlab # [ 7.776806] systemd[1]: Reached target Network.968gitlab # [ 7.783584] systemd[1]: Starting gitlab-workhorse.service...969gitlab # [ 7.785733] systemd[1]: Starting Nginx Web Server...970gitlab # [ 7.791784] systemd[1]: Starting PostgreSQL Server...971gitlab # [ 7.792773] dhcpcd[716]: dhcpcd-10.3.2 starting972gitlab # [ 7.795954] dhcpcd[767]: dev: loaded udev973gitlab # [ 7.798136] systemd[1]: Starting Redis Server - redis-gitlab...974gitlab # [ 8.034146] 8021q: 802.1Q VLAN Support v1.8975gitlab # [ 8.035086] 8021q: adding VLAN 0 to HW filter on device eth1976gitlab # [ 7.804462] systemd[1]: Starting Permit User Sessions...977gitlab # [ 7.820455] systemd[1]: Finished Permit User Sessions.978gitlab # [ 7.832182] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.979gitlab # [ 7.842154] systemd[1]: Started Getty on tty1.980gitlab # [ 7.844450] systemd[1]: Reached target Login Prompts.981gitlab # [ 8.160067] cfg80211: Loading compiled-in X.509 certificates for regulatory database982gitlab # [ 8.166064] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'983gitlab # [ 8.169087] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'984gitlab # [ 8.172526] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2985gitlab # [ 8.175532] cfg80211: failed to load regulatory.db986gitlab # [ 7.966245] nginx-pre-start[786]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok987gitlab # [ 7.967125] nginx-pre-start[786]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful988gitlab # [ 7.976439] systemd[1]: Started Nginx Web Server.989gitlab # [ 8.215989] 8021q: adding VLAN 0 to HW filter on device eth0990gitlab # [ 7.991575] dhcpcd[767]: eth0: waiting for carrier991gitlab # [ 7.992695] dhcpcd[767]: eth0: carrier acquired992gitlab # [ 7.996625] dhcpcd[767]: DUID 00:01:00:01:32:29:65:f7:52:54:00:12:34:56993gitlab # [ 8.003181] dhcpcd[767]: eth0: IAID 00:12:34:56994gitlab # [ 8.004243] dhcpcd[767]: eth0: adding address fe80::5054:ff:fe12:3456995gitlab # [ 8.024808] redis-server[789]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.996gitlab # [ 8.025951] redis-server[789]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo997gitlab # [ 8.027597] redis-server[789]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=789, just started998gitlab # [ 8.028575] redis-server[789]: Configuration loaded999gitlab # [ 8.031882] redis[789]: monotonic clock: POSIX clock_gettime1000gitlab # [ 8.032841] systemd[1]: Started Redis Server - redis-gitlab.1001gitlab # [ 8.033967] redis[789]: Running mode=standalone, port=0.1002gitlab # [ 8.034853] redis[789]: Server initialized1003gitlab # [ 8.035937] redis[789]: Ready to accept connections unix1004gitlab # [ 8.037114] redis[789]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1005gitlab # [ 8.050217] postgresql-pre-start[796]: The files belonging to this database system will be owned by user "postgres".1006gitlab # [ 8.050873] postgresql-pre-start[796]: This user must also own the server process.1007gitlab # [ 8.055336] postgresql-pre-start[796]: The database cluster will be initialized with locale "en_US.UTF-8".1008gitlab # [ 8.057975] postgresql-pre-start[796]: The default database encoding has accordingly been set to "UTF8".1009gitlab # [ 8.061560] postgresql-pre-start[796]: The default text search configuration will be set to "english".1010gitlab # [ 8.063821] postgresql-pre-start[796]: Data page checksums are enabled.1011gitlab # [ 8.065480] postgresql-pre-start[796]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok1012gitlab # [ 8.065843] postgresql-pre-start[796]: creating subdirectories ... ok1013gitlab # [ 8.066915] postgresql-pre-start[796]: selecting dynamic shared memory implementation ... posix1014gitlab # [ 8.102546] postgresql-pre-start[796]: selecting default "max_connections" ... 1001015gitlab # [ 8.147621] postgresql-pre-start[796]: selecting default "shared_buffers" ... 128MB1016gitlab # [ 8.330374] dhcpcd[767]: eth0: soliciting a DHCP lease1017gitlab # [ 8.568290] NET: Registered PF_PACKET protocol family1018gitlab # [ 8.345890] dhcpcd[767]: eth0: offered 10.0.2.15 from 10.0.2.21019gitlab # [ 8.350370] dhcpcd[767]: eth0: probing address 10.0.2.15/241020gitlab # [ 8.414170] postgresql-pre-start[796]: selecting default time zone ... UTC1021gitlab # [ 8.416674] postgresql-pre-start[796]: creating configuration files ... ok1022gitlab # [ 8.446391] systemd[1]: Started gitlab-workhorse.service.1023gitlab # [ 8.602119] postgresql-pre-start[796]: running bootstrap script ... ok1024gitlab # [ 8.663721] gitlab-workhorse[826]: time="2026-09-01T10:31:19Z" level=info msg=Starting build_time=19700101.000000 version=19.0.41025gitlab # [ 8.664698] gitlab-workhorse[826]: time="2026-09-01T10:31:19Z" level=info msg="Using redis/go-redis"1026gitlab # [ 8.665898] gitlab-workhorse[826]: time="2026-09-01T10:31:19Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1027gitlab # [ 9.128476] postgresql-pre-start[796]: performing post-bootstrap initialization ... ok1028gitlab # [ 9.691887] postgresql-pre-start[796]: syncing data to disk ... ok1029gitlab # [ 9.692729] postgresql-pre-start[796]: initdb: warning: enabling "trust" authentication for local connections1030gitlab # [ 9.693936] postgresql-pre-start[796]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1031gitlab # [ 9.695061] postgresql-pre-start[796]: Success. You can now start the database server using:1032gitlab # [ 9.695844] postgresql-pre-start[796]: pg_ctl -D /var/lib/postgresql/18 -l logfile start1033gitlab # [ 9.807282] postgres[858]: [858] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1034gitlab # [ 9.809966] postgres[858]: [858] LOG: listening on IPv6 address "::1", port 54321035gitlab # [ 9.810833] postgres[858]: [858] LOG: listening on IPv4 address "127.0.0.1", port 54321036gitlab # [ 9.815429] postgres[858]: [858] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1037gitlab # [ 9.827946] postgres[868]: [868] LOG: database system was shut down at 2026-09-01 10:31:20 GMT1038gitlab # [ 9.835976] postgres[858]: [858] LOG: database system is ready to accept connections1039gitlab # [ 9.844635] systemd[1]: Started PostgreSQL Server.1040gitlab # [ 9.847552] systemd[1]: Starting PostgreSQL Setup Scripts...1041gitlab # [ 10.020211] postgresql-setup-start[883]: CREATE DATABASE1042gitlab # [ 10.040466] postgresql-setup-start[889]: CREATE ROLE1043gitlab # [ 10.050782] postgresql-setup-start[891]: ALTER DATABASE1044gitlab # [ 10.054811] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[878]: mkdir -p /var/gitlab/state/home/.ssh: OK1045gitlab # [ 10.058534] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[878]: chmod 700 /var/gitlab/state/home/.ssh: OK1046gitlab # [ 10.073204] postgresql-setup-start[900]: CREATE ROLE1047gitlab # [ 10.076446] systemd[1]: Finished PostgreSQL Setup Scripts.1048gitlab # [ 10.077866] systemd[1]: Reached target PostgreSQL.1049gitlab # [ 10.079441] systemd[1]: Starting gitlab-postgresql.service...1050gitlab # [ 10.089752] dhcpcd[767]: eth0: soliciting an IPv6 router1051gitlab # [ 10.091151] dhcpcd[767]: eth0: Router Advertisement from fe80::21052gitlab # [ 10.092624] dhcpcd[767]: eth0: adding address fec0::5054:ff:fe12:3456/641053gitlab # [ 10.093124] dhcpcd[767]: eth0: adding route to fec0::/641054gitlab # [ 10.093881] dhcpcd[767]: eth0: adding default route via fe80::21055gitlab # [ 10.099188] systemd[1]: Finished gitlab-config.service.1056gitlab # [ 10.102610] systemd[1]: Started gitaly.service.1057gitlab # [ 10.104852] systemd[1]: Starting gitlab-db-config.service...1058gitlab # [ 10.189288] gitlab-postgresql-start[918]: CREATE DATABASE1059gitlab # [ 10.223088] gitlab-postgresql-start[925]: CREATE EXTENSION1060gitlab # [ 10.309512] gitlab-postgresql-start[927]: CREATE EXTENSION1061gitlab # [ 10.314357] systemd[1]: Finished gitlab-postgresql.service.1062gitlab # [ 10.739373] gitaly[916]: time="2026-09-01T10:31:21.896Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=9161063gitlab # [ 10.740283] gitaly[916]: time="2026-09-01T10:31:21.897Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=9161064gitlab # [ 10.741509] gitaly[916]: time="2026-09-01T10:31:21.897Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=9161065gitlab # [ 10.742926] gitaly[916]: time="2026-09-01T10:31:21.897Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=9161066gitlab # [ 10.743963] gitaly[916]: time="2026-09-01T10:31:21.897Z" level=info msg="Starting Gitaly" pid=916 version=1067gitlab # [ 11.002729] gitaly[916]: time="2026-09-01T10:31:22.160Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=262 pid=9161068gitlab # [ 11.004273] gitaly[916]: time="2026-09-01T10:31:22.160Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=9161069gitlab # [ 11.005539] gitaly[916]: time="2026-09-01T10:31:22.161Z" level=info msg="finished initializing command factory" duration_ms=0 pid=9161070gitlab # [ 11.006823] gitaly[916]: time="2026-09-01T10:31:22.161Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-916/git-exec-3400960224.d/git pid=9161071gitlab # [ 11.014282] gitaly[916]: time="2026-09-01T10:31:22.172Z" level=info msg="finished detecting git version" duration_ms=10 pid=9161072gitlab # [ 11.015121] gitaly[916]: time="2026-09-01T10:31:22.172Z" level=info msg="using Git version" pid=916 version=2.53.g1073gitlab # [ 11.020627] gitaly[916]: time="2026-09-01T10:31:22.178Z" level=info msg="clearing disk cache object folder" pid=916 storage=default1074gitlab # [ 11.021891] gitaly[916]: time="2026-09-01T10:31:22.179Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1543602304 pid=916 storage=default1075gitlab # [ 11.023298] gitaly[916]: time="2026-09-01T10:31:22.179Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1543602304 pid=916 storage=default1076gitlab # [ 11.024970] gitaly[916]: time="2026-09-01T10:31:22.179Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=9161077gitlab # [ 11.026395] gitaly[916]: time="2026-09-01T10:31:22.179Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=9161078gitlab # [ 11.027949] gitaly[916]: time="2026-09-01T10:31:22.179Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1543602304 clear_duration_ms=0 pid=916 storage=default1079gitlab # [ 11.143289] gitaly[916]: time="2026-09-01T10:31:22.300Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=9161080gitlab # [ 11.144143] gitaly[916]: time="2026-09-01T10:31:22.300Z" level=info msg="counting repositories" pid=9161081gitlab # [ 11.144778] gitaly[916]: time="2026-09-01T10:31:22.301Z" level=info msg="starting to count repositories" pid=916 storage_path=/var/gitlab/state/repositories1082gitlab # [ 11.145930] gitaly[916]: time="2026-09-01T10:31:22.301Z" level=info msg="finished tempdir cleaner walk" pid=916 storage=default time_ms=01083gitlab # [ 11.147335] gitaly[916]: time="2026-09-01T10:31:22.301Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=916 storage_path=/var/gitlab/state/repositories1084gitlab # [ 11.148985] gitaly[916]: time="2026-09-01T10:31:22.301Z" level=info msg="completed counting all repositories" counting_duration=0 pid=9161085gitlab # [ 11.150964] gitaly[916]: time="2026-09-01T10:31:22.300Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=916 schema=unix1086gitlab # [ 11.152297] gitaly[916]: time="2026-09-01T10:31:22.301Z" level=info msg="listening at address" address=/run/gitaly/gitaly-916/sock.d/intern pid=916 schema=unix1087gitlab # [ 11.167616] gitaly[916]: time="2026-09-01T10:31:22.325Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9161088gitlab # [ 11.169104] gitaly[916]: time="2026-09-01T10:31:22.325Z" level=info msg="Started Gitaly" duration_ms=428 pid=9161089gitlab # [ 11.179705] gitaly[916]: time="2026-09-01T10:31:22.336Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9161090gitlab # [ 11.182851] gitaly[916]: time="2026-09-01T10:31:22.336Z" level=info msg="maintenance: daily scheduled" pid=916 scheduled="2026-09-01 12:00:00 +0000 UTC"1091gitlab # [ 12.161437] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[921]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1092gitlab # [ 12.162464] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[921]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1093gitlab # [ 12.181416] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[921]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1094gitlab # [ 13.521849] dhcpcd[767]: eth0: leased 10.0.2.15 for 86400 seconds1095gitlab # [ 13.522493] dhcpcd[767]: eth0: adding route to 10.0.2.0/241096gitlab # [ 13.524037] dhcpcd[767]: eth0: adding default route via 10.0.2.21097gitlab # [ 13.587146] systemd[1]: Started DHCP Client.1098gitlab # [ 13.589144] systemd[1]: Reached target Network is Online.1099gitlab # [ 15.304743] gitaly[916]: time="2026-09-01T10:31:26.462Z" level=info msg="License database preloaded" duration_ms=4564 pid=9161100gitlab # [ 52.960173] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[921]: W, [2026-09-01T10:32:04.117854 #921] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1101gitlab # [ 55.526095] postgres[1427]: [1427] ERROR: relation "feature_gates" does not exist at character 5231102gitlab # [ 55.526400] postgres[1427]: [1427] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1103gitlab # [ 55.526984] postgres[1427]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1104gitlab # [ 55.528295] postgres[1427]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1105gitlab # [ 55.529296] postgres[1427]: attidentity AS identity,1106gitlab # [ 55.530215] postgres[1427]: attgenerated as attgenerated1107gitlab # [ 55.531343] postgres[1427]: FROM pg_attribute a1108gitlab # [ 55.532203] postgres[1427]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1109gitlab # [ 55.533761] postgres[1427]: LEFT JOIN pg_type t ON a.atttypid = t.oid1110gitlab # [ 55.534906] postgres[1427]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1111gitlab # [ 55.536048] postgres[1427]: WHERE a.attrelid = '"feature_gates"'::regclass1112gitlab # [ 55.536930] postgres[1427]: AND a.attnum > 0 AND NOT a.attisdropped1113gitlab # [ 55.538112] postgres[1427]: ORDER BY a.attnum1114gitlab # [ 55.538984] postgres[1427]:1115gitlab # [ 58.728549] gitlab-workhorse[826]: time="2026-09-01T10:32:09Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1116gitlab # [ 59.839204] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1469]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1117gitlab # [ 59.839788] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1469]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1118gitlab # [ 68.738400] gitlab-workhorse[826]: time="2026-09-01T10:32:19Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1119gitlab # [ 78.746403] gitlab-workhorse[826]: time="2026-09-01T10:32:29Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1120gitlab # [ 78.992641] postgres[1563]: [1563] ERROR: relation "table_without_models" does not exist1121gitlab # [ 78.993176] postgres[1563]: [1563] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1122gitlab # [ 88.756282] gitlab-workhorse[826]: time="2026-09-01T10:32:39Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1123gitlab # [ 95.501378] systemd[1]: Finished gitlab-db-config.service.1124gitlab # [ 95.504348] systemd[1]: Started gitlab-sidekiq.service.1125gitlab # [ 95.506667] systemd[1]: Starting gitlab.service...1126gitlab # [ 95.961778] bundle[1793]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1127gitlab # [ 95.962830] bundle[1793]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1128gitlab # [ 95.969380] bundle[1793]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1129gitlab # [ 98.757061] gitlab-workhorse[826]: time="2026-09-01T10:32:49Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1130gitlab # [ 99.659687] sidekiq-cluster[1792]: {"severity":"INFO","time":"2026-09-01T10:32:50.791Z","message":"Starting cluster with 1 processes"}1131gitlab # [ 99.852802] bundle[1793]: {"timestamp":"2026-09-01T10:32:50.983Z","pid":1793,"message":"* Enabling systemd notification integration"}1132gitlab # [ 99.853698] bundle[1793]: {"timestamp":"2026-09-01T10:32:51.010Z","pid":1793,"message":"Puma starting in cluster mode..."}1133gitlab # [ 99.854295] bundle[1793]: {"timestamp":"2026-09-01T10:32:51.010Z","pid":1793,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1134gitlab # [ 99.854989] bundle[1793]: {"timestamp":"2026-09-01T10:32:51.010Z","pid":1793,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1135gitlab # [ 99.855956] bundle[1793]: {"timestamp":"2026-09-01T10:32:51.010Z","pid":1793,"message":"* Min threads: 0"}1136gitlab # [ 99.858607] bundle[1793]: {"timestamp":"2026-09-01T10:32:51.011Z","pid":1793,"message":"* Max threads: 4"}1137gitlab # [ 99.865334] bundle[1793]: {"timestamp":"2026-09-01T10:32:51.011Z","pid":1793,"message":"* Environment: production"}1138gitlab # [ 99.867221] bundle[1793]: {"timestamp":"2026-09-01T10:32:51.023Z","pid":1793,"message":"* Master PID: 1793"}1139gitlab # [ 99.869112] bundle[1793]: {"timestamp":"2026-09-01T10:32:51.025Z","pid":1793,"message":"* Workers: 2"}1140gitlab # [ 99.869859] bundle[1793]: {"timestamp":"2026-09-01T10:32:51.027Z","pid":1793,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1141gitlab # [ 99.870857] bundle[1793]: {"timestamp":"2026-09-01T10:32:51.027Z","pid":1793,"message":"* Preloading application"}1142gitlab # [ 108.766177] gitlab-workhorse[826]: time="2026-09-01T10:32:59Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1143gitlab # [ 118.776295] gitlab-workhorse[826]: time="2026-09-01T10:33:09Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1144gitlab # [ 128.778207] gitlab-workhorse[826]: time="2026-09-01T10:33:19Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1145gitlab # [ 131.010765] bundle[1793]: W, [2026-09-01T10:33:22.168469 #1793] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1146gitlab # [ 134.000127] sidekiq-cluster[1832]: 2026-09-01T10:33:25.157Z pid=1832 tid=wow INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1832, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>449339392}1147gitlab # [ 134.238953] sidekiq-cluster[1832]: W, [2026-09-01T10:33:25.396884 #1832] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1148gitlab # [ 134.295992] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:25.453Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1149gitlab # [ 136.886220] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.043Z","message":"GitLab reliable fetch activated!"}1150gitlab # [ 136.886517] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.044Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1151gitlab # [ 136.925241] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.083Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1152gitlab # [ 136.928799] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.086Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1153gitlab # [ 136.932409] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.090Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1154gitlab # [ 136.935824] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.093Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1155gitlab # [ 136.939810] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.097Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1156gitlab # [ 136.944182] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.102Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1157gitlab # [ 136.948094] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.106Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1158gitlab # [ 136.954612] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.112Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1159gitlab # [ 136.961367] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.119Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1160gitlab # [ 136.968459] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.126Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1161gitlab # [ 136.972132] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.130Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1162gitlab # [ 136.976473] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.134Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1163gitlab # [ 136.979890] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.137Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1164gitlab # [ 136.982860] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.141Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1165gitlab # [ 136.986635] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.144Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1166gitlab # [ 136.989770] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.147Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1167gitlab # [ 136.995161] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.153Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1168gitlab # [ 137.001743] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.159Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1169gitlab # [ 137.010046] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.168Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1170gitlab # [ 137.014675] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.172Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1171gitlab # [ 137.019434] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.176Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1172gitlab # [ 137.027951] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.185Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1173gitlab # [ 137.032697] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.190Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1174gitlab # [ 137.036571] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.194Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1175gitlab # [ 137.040336] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.198Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1176gitlab # [ 137.047799] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.205Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1177gitlab # [ 137.054884] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.212Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1178gitlab # [ 137.061115] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.219Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1179gitlab # [ 137.067726] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.225Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1180gitlab # [ 137.075207] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.233Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1181gitlab # [ 137.078147] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.236Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1182gitlab # [ 137.081925] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.240Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1183gitlab # [ 137.085879] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.243Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1184gitlab # [ 137.092707] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.250Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1185gitlab # [ 137.099698] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.257Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1186gitlab # [ 137.106449] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.264Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1187gitlab # [ 137.110620] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.268Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1188gitlab # [ 137.115172] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.273Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1189gitlab # [ 137.118628] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.276Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1190gitlab # [ 137.122661] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.280Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1191gitlab # [ 137.126378] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.284Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1192gitlab # [ 137.131429] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.288Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1193gitlab # [ 137.135249] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.293Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1194gitlab # [ 137.138872] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.296Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1195gitlab # [ 137.143614] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.301Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1196gitlab # [ 137.147244] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.305Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1197gitlab # [ 137.151517] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.309Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1198gitlab # [ 137.155758] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.313Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1199gitlab # [ 137.160253] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.318Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1200gitlab # [ 137.167851] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.325Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1201gitlab # [ 137.208734] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.365Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1202gitlab # [ 137.212945] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.371Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1203gitlab # [ 137.222468] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.380Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1204gitlab # [ 137.227277] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.385Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1205gitlab # [ 137.231354] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.389Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1206gitlab # [ 137.234301] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.392Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1207gitlab # [ 137.238659] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.396Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1208gitlab # [ 137.242548] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.400Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1209gitlab # [ 137.246493] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.404Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1210gitlab # [ 137.249283] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.407Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1211gitlab # [ 137.253148] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.411Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1212gitlab # [ 137.255777] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.413Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1213gitlab # [ 137.261150] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.419Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1214gitlab # [ 137.265210] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.423Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1215gitlab # [ 137.269637] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.427Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1216gitlab # [ 137.274102] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.432Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1217gitlab # [ 137.277256] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.435Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1218gitlab # [ 137.282133] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.440Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1219gitlab # [ 137.285967] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.444Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1220gitlab # [ 137.288715] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.446Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1221gitlab # [ 137.292480] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.450Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1222gitlab # [ 137.295206] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.453Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1223gitlab # [ 137.298660] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.456Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1224gitlab # [ 137.301055] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.459Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1225gitlab # [ 137.303940] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.462Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1226gitlab # [ 137.308074] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.466Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1227gitlab # [ 137.310775] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.468Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1228gitlab # [ 137.314707] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.472Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1229gitlab # [ 137.317502] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.475Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1230gitlab # [ 137.320612] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.478Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1231gitlab # [ 137.325135] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.483Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1232gitlab # [ 137.328599] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.486Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1233gitlab # [ 137.334318] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.492Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1234gitlab # [ 137.337342] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.495Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1235gitlab # [ 137.341166] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.499Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1236gitlab # [ 137.344538] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.502Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1237gitlab # [ 137.347839] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.505Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1238gitlab # [ 137.351139] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.509Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1239gitlab # [ 137.355475] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.513Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1240gitlab # [ 137.359255] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.517Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1241gitlab # [ 137.362384] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.520Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1242gitlab # [ 137.365619] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.523Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1243gitlab # [ 137.369374] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.527Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1244gitlab # [ 137.372749] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.530Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1245gitlab # [ 137.378406] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.536Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1246gitlab # [ 137.382334] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.540Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1247gitlab # [ 137.386082] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.543Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1248gitlab # [ 137.390125] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.545Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1249gitlab # [ 137.398904] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.557Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1250gitlab # [ 137.401604] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.559Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1251gitlab # [ 137.404883] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.563Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1252gitlab # [ 137.409560] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.567Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1253gitlab # [ 137.413093] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.571Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1254gitlab # [ 137.416944] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.575Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1255gitlab # [ 137.421275] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.579Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1256gitlab # [ 137.425391] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.583Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1257gitlab # [ 137.428711] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.586Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1258gitlab # [ 137.433092] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.591Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1259gitlab # [ 137.437379] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:33:28.595Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1260gitlab # [ 138.779910] gitlab-workhorse[826]: time="2026-09-01T10:33:29Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1261gitlab # [ 148.788284] gitlab-workhorse[826]: time="2026-09-01T10:33:39Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1262gitlab # [ 158.798674] gitlab-workhorse[826]: time="2026-09-01T10:33:49Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1263gitlab # [ 167.816929] postgres[2460]: [2460] ERROR: relation "table_without_models" does not exist1264gitlab # [ 167.818059] postgres[2460]: [2460] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1265gitlab # [ 168.808339] gitlab-workhorse[826]: time="2026-09-01T10:33:59Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1266gitlab # [ 169.492914] postgres[2475]: [2475] ERROR: relation "table_without_models" does not exist1267gitlab # [ 169.494794] postgres[2475]: [2475] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1268gitlab # [ 171.505531] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:02.662Z","message":"Booted Rails 7.2.3.1 application in production environment"}1269gitlab # [ 171.508673] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:02.666Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1270gitlab # [ 171.511721] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:02.666Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1271gitlab # [ 171.514408] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:02.667Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1272gitlab # [ 171.517262] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:02.671Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1273gitlab # [ 173.307898] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:04.465Z","message":"Cleaning working queues"}1274gitlab # [ 173.310515] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:04.468Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"09c9560af29d98bf3563deed","created_at":"2026-09-01T10:32:46.306Z","trace_propagation_headers":{"sentry-trace":"af3f6df118d742808b2e588b92dcf9d3-081e6fa5f9524ad9","baggage":"sentry-trace_id=af3f6df118d742808b2e588b92dcf9d3,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"5e9dc790d8624042bafa8602328f3877","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-01T10:32:46.316Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-09c9560af29d98bf3563deed: start","job_status":"start","queue_duration_s":78.151458,"scheduling_latency_s":78.151523}1275gitlab # [ 173.550401] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:04.707Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"09c9560af29d98bf3563deed","created_at":"2026-09-01T10:32:46.306Z","trace_propagation_headers":{"sentry-trace":"af3f6df118d742808b2e588b92dcf9d3-081e6fa5f9524ad9","baggage":"sentry-trace_id=af3f6df118d742808b2e588b92dcf9d3,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"5e9dc790d8624042bafa8602328f3877","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-01T10:32:46.316Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-09c9560af29d98bf3563deed: done: 0.239927 sec","job_status":"done","queue_duration_s":78.151458,"scheduling_latency_s":78.151523,"redis_calls":30,"redis_duration_s":0.028793,"redis_read_bytes":410,"redis_write_bytes":3701,"redis_feature_flag_calls":19,"redis_feature_flag_duration_s":0.021147,"redis_feature_flag_read_bytes":400,"redis_feature_flag_write_bytes":2454,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.006288,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001358,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.106698,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.09524422799999999,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.239927,"completed_at":"2026-09-01T10:34:04.707Z","load_balancing_strategy":"primary","db_duration_s":0.068836,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1276gitlab # [ 173.612419] bundle[1793]: {"timestamp":"2026-09-01T10:34:04.770Z","pid":1793,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1277gitlab # [ 173.615274] bundle[1793]: {"timestamp":"2026-09-01T10:34:04.770Z","pid":1793,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1278gitlab # [ 173.617642] bundle[1793]: {"timestamp":"2026-09-01T10:34:04.770Z","pid":1793,"message":"! #\u003cThread:0x00007e0894103b80 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1279gitlab # [ 173.618073] bundle[1793]: {"timestamp":"2026-09-01T10:34:04.770Z","pid":1793,"message":"! #\u003cThread:0x00007e0894381128 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1280gitlab # [ 173.618801] bundle[1793]: {"timestamp":"2026-09-01T10:34:04.770Z","pid":1793,"message":"Use Ctrl-C to stop"}1281gitlab # [ 173.619754] bundle[1793]: {"timestamp":"2026-09-01T10:34:04.770Z","pid":1793,"message":"Sending status to systemd every 1.0 sec"}1282gitlab # [ 173.952370] bundle[1793]: {"timestamp":"2026-09-01T10:34:05.110Z","pid":1793,"message":"- Worker 0 (PID: 2553) booted in 0.3s, phase: 0"}1283gitlab # [ 173.980698] bundle[1793]: {"timestamp":"2026-09-01T10:34:05.138Z","pid":1793,"message":"- Worker 1 (PID: 2557) booted in 0.31s, phase: 0"}1284gitlab # [ 173.985318] systemd[1]: Started gitlab.service.1285gitlab # [ 173.986690] systemd[1]: Reached target Common target for all GitLab services..1286gitlab # [ 173.988578] systemd[1]: Starting setup-gitlab.service...1287gitlab # [ 174.006280] setup-gitlab-start[2581]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1288gitlab # [ 176.535827] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:34:07 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 25131289gitlab # [ 176.539625] setup-gitlab-start[2581]: + TOKEN=nixbot-test-token-of-20-chars1290gitlab # [ 176.541264] setup-gitlab-start[2581]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1291gitlab # [ 176.552488] runuser[2599]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1292gitlab # [ 177.022493] setup-gitlab-start[2600]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1293gitlab # [ 177.023059] setup-gitlab-start[2600]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1294gitlab # [ 177.024376] setup-gitlab-start[2600]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1295gitlab # [ 177.784249] setup-gitlab-start[2600]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1296gitlab # [ 177.787446] setup-gitlab-start[2600]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1297gitlab # [ 177.790206] setup-gitlab-start[2600]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1298gitlab # [ 184.068933] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:15.223Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"d54692b102d22833d741801a","created_at":"2026-09-01T10:34:15.195Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"577a30ffe04224184d26231845cbcfaf","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.218Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-d54692b102d22833d741801a: start","job_status":"start","queue_duration_s":0.004333,"scheduling_latency_s":0.004379}1299gitlab # [ 184.158412] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:15.316Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"d54692b102d22833d741801a","created_at":"2026-09-01T10:34:15.195Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"577a30ffe04224184d26231845cbcfaf","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.218Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-d54692b102d22833d741801a: done: 0.092814 sec","job_status":"done","queue_duration_s":0.004333,"scheduling_latency_s":0.004379,"redis_calls":12,"redis_duration_s":0.02136,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004701,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009312,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.007347,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.004669588,"exclusive_lock_hold_duration_s":0.017346059,"duration_s":0.092814,"completed_at":"2026-09-01T10:34:15.315Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0177,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1300gitlab # [ 184.234603] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:15.392Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"606213aad3798f3744cbf659","created_at":"2026-09-01T10:34:15.269Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f4f4673e6f363e180f04a5b6e6b88eb1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-01T10:34:15.390Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-606213aad3798f3744cbf659: start","job_status":"start","queue_duration_s":0.002348,"scheduling_latency_s":0.002388}1301gitlab # [ 184.267727] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:15.425Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1302gitlab # [ 184.282650] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:15.440Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"606213aad3798f3744cbf659","created_at":"2026-09-01T10:34:15.269Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f4f4673e6f363e180f04a5b6e6b88eb1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-01T10:34:15.390Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-606213aad3798f3744cbf659: done: 0.048118 sec","job_status":"done","queue_duration_s":0.002348,"scheduling_latency_s":0.002388,"redis_calls":10,"redis_duration_s":0.011951,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002378,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005956,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003617,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001730388,"duration_s":0.048118,"completed_at":"2026-09-01T10:34:15.440Z","load_balancing_strategy":"primary","db_duration_s":0.000942,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1303gitlab # [ 184.334199] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:15.491Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"c886f1145596f3b7761e3b5e","created_at":"2026-09-01T10:34:15.403Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0c01ff5b219a7a567ff3e65e2660ab45","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.417Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-c886f1145596f3b7761e3b5e: start","job_status":"start","queue_duration_s":0.07478,"scheduling_latency_s":0.074812}1304gitlab # [ 185.073469] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.230Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"c886f1145596f3b7761e3b5e","created_at":"2026-09-01T10:34:15.403Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0c01ff5b219a7a567ff3e65e2660ab45","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.417Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-c886f1145596f3b7761e3b5e: done: 0.738856 sec","job_status":"done","queue_duration_s":0.07478,"scheduling_latency_s":0.074812,"redis_calls":565,"redis_duration_s":0.370381,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000726,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004062,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.365593,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.393423,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000387759,"duration_s":0.738856,"completed_at":"2026-09-01T10:34:16.230Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000484,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1305gitlab # [ 185.130915] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.288Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"0b5a2d067f473fa3caaecb74","created_at":"2026-09-01T10:34:15.426Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"25d83b12550c1fc8b9d10ae02196b72f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.439Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-0b5a2d067f473fa3caaecb74: start","job_status":"start","queue_duration_s":0.849515,"scheduling_latency_s":0.849552}1306gitlab # [ 185.182185] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.339Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"0b5a2d067f473fa3caaecb74","created_at":"2026-09-01T10:34:15.426Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"25d83b12550c1fc8b9d10ae02196b72f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.439Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-0b5a2d067f473fa3caaecb74: done: 0.051298 sec","job_status":"done","queue_duration_s":0.849515,"scheduling_latency_s":0.849552,"redis_calls":12,"redis_duration_s":0.0040490000000000005,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000727,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001667,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001655,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020417,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000932521,"exclusive_lock_hold_duration_s":0.010548547,"duration_s":0.051298,"completed_at":"2026-09-01T10:34:16.339Z","load_balancing_strategy":"primary","db_duration_s":0.006264,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1307gitlab # [ 185.183687] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.341Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"0f931b9043c8d3fc761f40b3","created_at":"2026-09-01T10:34:15.509Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.521Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-0f931b9043c8d3fc761f40b3: start","job_status":"start","queue_duration_s":0.820575,"scheduling_latency_s":0.820598}1308gitlab # [ 185.296052] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.453Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"0f931b9043c8d3fc761f40b3","created_at":"2026-09-01T10:34:15.509Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.521Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-0f931b9043c8d3fc761f40b3: done: 0.112077 sec","job_status":"done","queue_duration_s":0.820575,"scheduling_latency_s":0.820598,"redis_calls":71,"redis_duration_s":0.018123,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002475,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001078,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.013894,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000676,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.088289,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000372115,"duration_s":0.112077,"completed_at":"2026-09-01T10:34:16.453Z","load_balancing_strategy":"primary","db_duration_s":0.004061,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1309gitlab # [ 185.349269] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.506Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"40a3eb29bdfda30d1d25a550","created_at":"2026-09-01T10:34:15.531Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b30da8041259a7e8159f372aa0d5c5f2","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.538Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-40a3eb29bdfda30d1d25a550: start","job_status":"start","queue_duration_s":0.968723,"scheduling_latency_s":0.968763}1310gitlab # [ 185.411269] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.568Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"40a3eb29bdfda30d1d25a550","created_at":"2026-09-01T10:34:15.531Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b30da8041259a7e8159f372aa0d5c5f2","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.538Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-40a3eb29bdfda30d1d25a550: done: 0.061889 sec","job_status":"done","queue_duration_s":0.968723,"scheduling_latency_s":0.968763,"redis_calls":16,"redis_duration_s":0.004522,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002009,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001683,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00083,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027496,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000416254,"duration_s":0.061889,"completed_at":"2026-09-01T10:34:16.568Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009458,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1311gitlab # [ 185.412931] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.570Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"1a15c680e3f2a7314db9396d","created_at":"2026-09-01T10:34:15.547Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5802e1a3072fc4501d3d367d612fb25","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.556Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-1a15c680e3f2a7314db9396d: start","job_status":"start","queue_duration_s":1.014761,"scheduling_latency_s":1.01479}1312gitlab # [ 185.436649] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.594Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"1a15c680e3f2a7314db9396d","created_at":"2026-09-01T10:34:15.547Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5802e1a3072fc4501d3d367d612fb25","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.556Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-1a15c680e3f2a7314db9396d: done: 0.023596 sec","job_status":"done","queue_duration_s":1.014761,"scheduling_latency_s":1.01479,"redis_calls":12,"redis_duration_s":0.002862,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000503,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000439,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001457,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000463,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013497,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000364571,"duration_s":0.023596,"completed_at":"2026-09-01T10:34:16.594Z","load_balancing_strategy":"primary","db_duration_s":0.00086,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1313gitlab # [ 185.481254] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.596Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"8c4bb9985a9516d62294fd89","created_at":"2026-09-01T10:34:15.563Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d11dba87fc21be74aa6912148c150ed0","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.570Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-8c4bb9985a9516d62294fd89: start","job_status":"start","queue_duration_s":1.025325,"scheduling_latency_s":1.025357}1314gitlab # [ 185.500932] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.639Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"8c4bb9985a9516d62294fd89","created_at":"2026-09-01T10:34:15.563Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d11dba87fc21be74aa6912148c150ed0","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.570Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-8c4bb9985a9516d62294fd89: done: 0.042753 sec","job_status":"done","queue_duration_s":1.025325,"scheduling_latency_s":1.025357,"redis_calls":10,"redis_duration_s":0.002533,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000455,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001526,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000552,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017514,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000543924,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.042753,"completed_at":"2026-09-01T10:34:16.638Z","load_balancing_strategy":"primary","db_duration_s":0.016849,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1315gitlab # [ 185.584931] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.640Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"bc804ceae4539ff6a1e9ca4c","created_at":"2026-09-01T10:34:15.578Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3c2b9b34b63bb87e67ea6e45bff3ee1e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.589Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-bc804ceae4539ff6a1e9ca4c: start","job_status":"start","queue_duration_s":1.051196,"scheduling_latency_s":1.051222}1316gitlab # [ 185.585868] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.658Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1317gitlab # [ 185.586761] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.669Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"bc804ceae4539ff6a1e9ca4c","created_at":"2026-09-01T10:34:15.578Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3c2b9b34b63bb87e67ea6e45bff3ee1e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.589Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-bc804ceae4539ff6a1e9ca4c: done: 0.02833 sec","job_status":"done","queue_duration_s":1.051196,"scheduling_latency_s":1.051222,"redis_calls":10,"redis_duration_s":0.0026249999999999997,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000548,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00154,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000537,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015413,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000494756,"duration_s":0.02833,"completed_at":"2026-09-01T10:34:16.669Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003367,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1318gitlab # [ 185.629605] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.671Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"1a821dad3f17cc2c98d08380","created_at":"2026-09-01T10:34:15.607Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de64471cd9e72b9a4158eeb46aafba2e","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.616Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-1a821dad3f17cc2c98d08380: start","job_status":"start","queue_duration_s":1.054914,"scheduling_latency_s":1.054941}1319gitlab # [ 185.648038] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.695Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"1a821dad3f17cc2c98d08380","created_at":"2026-09-01T10:34:15.607Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de64471cd9e72b9a4158eeb46aafba2e","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.616Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-1a821dad3f17cc2c98d08380: done: 0.024141 sec","job_status":"done","queue_duration_s":1.054914,"scheduling_latency_s":1.054941,"redis_calls":12,"redis_duration_s":0.0026899999999999997,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000417,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001316,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000957,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013707,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000702883,"exclusive_lock_hold_duration_s":0.003231416,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.024141,"completed_at":"2026-09-01T10:34:16.695Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002042,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1320gitlab # [ 185.711053] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.697Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"1ad52276a3b64c4b0803ab9b","created_at":"2026-09-01T10:34:15.640Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7ff8abf6e9d94254140e7423fc39776b","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.650Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-1ad52276a3b64c4b0803ab9b: start","job_status":"start","queue_duration_s":1.04675,"scheduling_latency_s":1.046777}1321gitlab # [ 185.732338] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.727Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"1ad52276a3b64c4b0803ab9b","created_at":"2026-09-01T10:34:15.640Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7ff8abf6e9d94254140e7423fc39776b","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.650Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-1ad52276a3b64c4b0803ab9b: done: 0.03014 sec","job_status":"done","queue_duration_s":1.04675,"scheduling_latency_s":1.046777,"redis_calls":10,"redis_duration_s":0.0025050000000000003,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000531,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001525,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000449,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00037826,"duration_s":0.03014,"completed_at":"2026-09-01T10:34:16.727Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003418,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1322gitlab # [ 185.828791] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.729Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"74cf01b3c326bd0acb5f6c31","created_at":"2026-09-01T10:34:15.661Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e83ba3842152947554d9fda337dbfdd1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.670Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-74cf01b3c326bd0acb5f6c31: start","job_status":"start","queue_duration_s":1.058853,"scheduling_latency_s":1.058879}1323gitlab # [ 185.830039] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.743Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1324gitlab # [ 185.831092] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.753Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"74cf01b3c326bd0acb5f6c31","created_at":"2026-09-01T10:34:15.661Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e83ba3842152947554d9fda337dbfdd1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.670Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-74cf01b3c326bd0acb5f6c31: done: 0.024032 sec","job_status":"done","queue_duration_s":1.058853,"scheduling_latency_s":1.058879,"redis_calls":10,"redis_duration_s":0.002646,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000484,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0016,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000562,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013837,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00042715,"duration_s":0.024032,"completed_at":"2026-09-01T10:34:16.752Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000567,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1325gitlab # [ 185.864984] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.754Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"3417c5a1d01ad913e0f92a0b","created_at":"2026-09-01T10:34:15.677Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f0fcc0d120c174d1772358012f95f877","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.685Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-3417c5a1d01ad913e0f92a0b: start","job_status":"start","queue_duration_s":1.069461,"scheduling_latency_s":1.069486}1326gitlab # [ 185.917710] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.779Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"3417c5a1d01ad913e0f92a0b","created_at":"2026-09-01T10:34:15.677Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f0fcc0d120c174d1772358012f95f877","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.685Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-3417c5a1d01ad913e0f92a0b: done: 0.02505 sec","job_status":"done","queue_duration_s":1.069461,"scheduling_latency_s":1.069486,"redis_calls":10,"redis_duration_s":0.002125,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000457,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00128,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000388,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013293,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000372673,"duration_s":0.02505,"completed_at":"2026-09-01T10:34:16.779Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004182,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1327gitlab # [ 185.934052] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.781Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"d707dcf70fd83fe328a5dc42","created_at":"2026-09-01T10:34:15.700Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a24c6c7a76905cc3145dd096279119df","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.708Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-d707dcf70fd83fe328a5dc42: start","job_status":"start","queue_duration_s":1.073095,"scheduling_latency_s":1.073121}1328gitlab # [ 185.953482] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.844Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"d707dcf70fd83fe328a5dc42","created_at":"2026-09-01T10:34:15.700Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a24c6c7a76905cc3145dd096279119df","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.708Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-d707dcf70fd83fe328a5dc42: done: 0.062877 sec","job_status":"done","queue_duration_s":1.073095,"scheduling_latency_s":1.073121,"redis_calls":14,"redis_duration_s":0.003325,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000781,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001518,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001026,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042679,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000487492,"exclusive_lock_hold_duration_s":0.039530443,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.062877,"completed_at":"2026-09-01T10:34:16.844Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012089,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1329gitlab # [ 186.060539] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.846Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"60e99fbe20fdb0590c08d040","created_at":"2026-09-01T10:34:15.718Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"26dff5e947c682e66168ce55b58c8c89","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.727Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-60e99fbe20fdb0590c08d040: start","job_status":"start","queue_duration_s":1.119411,"scheduling_latency_s":1.119437}1330gitlab # [ 186.061854] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.881Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"60e99fbe20fdb0590c08d040","created_at":"2026-09-01T10:34:15.718Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"26dff5e947c682e66168ce55b58c8c89","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.727Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-60e99fbe20fdb0590c08d040: done: 0.034933 sec","job_status":"done","queue_duration_s":1.119411,"scheduling_latency_s":1.119437,"redis_calls":19,"redis_duration_s":0.004131,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000915,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000315,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002243,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000658,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023428,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000411784,"duration_s":0.034933,"completed_at":"2026-09-01T10:34:16.881Z","load_balancing_strategy":"primary","db_duration_s":0.001895,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1331gitlab # [ 186.085824] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.883Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"dc98d1d2791953fec0038f3f","created_at":"2026-09-01T10:34:15.735Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11c1726a80bcc4f9f8c5cc95b19d6a03","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.744Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-dc98d1d2791953fec0038f3f: start","job_status":"start","queue_duration_s":1.138783,"scheduling_latency_s":1.138809}1332gitlab # [ 186.104492] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.895Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1333gitlab # [ 186.108765] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.903Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"dc98d1d2791953fec0038f3f","created_at":"2026-09-01T10:34:15.735Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11c1726a80bcc4f9f8c5cc95b19d6a03","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.744Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-dc98d1d2791953fec0038f3f: done: 0.019919 sec","job_status":"done","queue_duration_s":1.138783,"scheduling_latency_s":1.138809,"redis_calls":10,"redis_duration_s":0.002324,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000485,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001329,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00051,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011622,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000359543,"duration_s":0.019919,"completed_at":"2026-09-01T10:34:16.902Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000558,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1334gitlab # [ 186.177545] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.904Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"a791138e843d3a9d1c8a0af7","created_at":"2026-09-01T10:34:15.752Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e049e58f22e747a1e0477797c9b11957","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.763Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-a791138e843d3a9d1c8a0af7: start","job_status":"start","queue_duration_s":1.141304,"scheduling_latency_s":1.141327}1335gitlab # [ 186.178639] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.936Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"a791138e843d3a9d1c8a0af7","created_at":"2026-09-01T10:34:15.752Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e049e58f22e747a1e0477797c9b11957","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.763Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-a791138e843d3a9d1c8a0af7: done: 0.03201 sec","job_status":"done","queue_duration_s":1.141304,"scheduling_latency_s":1.141327,"redis_calls":13,"redis_duration_s":0.002825,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000399,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001409,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.009703188000000001,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.03201,"completed_at":"2026-09-01T10:34:16.936Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002589,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1336gitlab # [ 186.238186] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.938Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"ae5fe76150c5092db7fd78dc","created_at":"2026-09-01T10:34:15.785Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"58009df06afd29fbea3c93e799908e7e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.792Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-ae5fe76150c5092db7fd78dc: start","job_status":"start","queue_duration_s":1.145517,"scheduling_latency_s":1.145541}1337gitlab # [ 186.297107] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.975Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"ae5fe76150c5092db7fd78dc","created_at":"2026-09-01T10:34:15.785Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"58009df06afd29fbea3c93e799908e7e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.792Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-ae5fe76150c5092db7fd78dc: done: 0.036813 sec","job_status":"done","queue_duration_s":1.145517,"scheduling_latency_s":1.145541,"redis_calls":12,"redis_duration_s":0.002876,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000393,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001237,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001246,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000724673,"exclusive_lock_hold_duration_s":0.011690312,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.036813,"completed_at":"2026-09-01T10:34:16.975Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009486,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1338gitlab # [ 186.315296] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:16.977Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"3a656868eff9f5ba371ea69f","created_at":"2026-09-01T10:34:15.804Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"56ef67939f76fa1d6896db3dc613c045","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.813Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-3a656868eff9f5ba371ea69f: start","job_status":"start","queue_duration_s":1.16337,"scheduling_latency_s":1.163397}1339gitlab # [ 186.336177] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.001Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"3a656868eff9f5ba371ea69f","created_at":"2026-09-01T10:34:15.804Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"56ef67939f76fa1d6896db3dc613c045","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.813Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-3a656868eff9f5ba371ea69f: done: 0.024843 sec","job_status":"done","queue_duration_s":1.16337,"scheduling_latency_s":1.163397,"redis_calls":12,"redis_duration_s":0.002666,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00084,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001414,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000412,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014881,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000416254,"duration_s":0.024843,"completed_at":"2026-09-01T10:34:17.001Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001028,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1340gitlab # [ 186.418232] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.003Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"80ff4d327d4e999a08434448","created_at":"2026-09-01T10:34:15.831Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"536887c0b7f6eaed50dcfc2987e82dcb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.842Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-80ff4d327d4e999a08434448: start","job_status":"start","queue_duration_s":1.16108,"scheduling_latency_s":1.161106}1341gitlab # [ 186.419266] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.018Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1342gitlab # [ 186.420170] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.026Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"80ff4d327d4e999a08434448","created_at":"2026-09-01T10:34:15.831Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"536887c0b7f6eaed50dcfc2987e82dcb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.842Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-80ff4d327d4e999a08434448: done: 0.022342 sec","job_status":"done","queue_duration_s":1.16108,"scheduling_latency_s":1.161106,"redis_calls":10,"redis_duration_s":0.00261,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00047,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001394,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000746,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013316,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00044419,"duration_s":0.022342,"completed_at":"2026-09-01T10:34:17.026Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000516,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1343gitlab # [ 186.469483] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.028Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"d2d846c1ad581ae41a41e569","created_at":"2026-09-01T10:34:15.862Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b66b0575fdd4cd62eeb93b493a2c68ce","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-01T10:34:15.873Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-d2d846c1ad581ae41a41e569: start","job_status":"start","queue_duration_s":1.155048,"scheduling_latency_s":1.155072}1344gitlab # [ 186.532080] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.039Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1345gitlab # [ 186.533148] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.049Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"d2d846c1ad581ae41a41e569","created_at":"2026-09-01T10:34:15.862Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b66b0575fdd4cd62eeb93b493a2c68ce","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-01T10:34:15.873Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-d2d846c1ad581ae41a41e569: done: 0.021642 sec","job_status":"done","queue_duration_s":1.155048,"scheduling_latency_s":1.155072,"redis_calls":10,"redis_duration_s":0.002195,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000398,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001361,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000436,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000672153,"duration_s":0.021642,"completed_at":"2026-09-01T10:34:17.049Z","load_balancing_strategy":"primary","db_duration_s":0.000449,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1346gitlab # [ 186.543255] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.052Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"04a8d88c5cef54340b11158f","created_at":"2026-09-01T10:34:15.889Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"03377e0e5c62e4094aa1160fc566f3b2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.899Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-04a8d88c5cef54340b11158f: start","job_status":"start","queue_duration_s":1.152743,"scheduling_latency_s":1.152775}1347gitlab # [ 186.561939] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.284Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"04a8d88c5cef54340b11158f","created_at":"2026-09-01T10:34:15.889Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"03377e0e5c62e4094aa1160fc566f3b2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.899Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-04a8d88c5cef54340b11158f: done: 0.232794 sec","job_status":"done","queue_duration_s":1.152743,"scheduling_latency_s":1.152775,"redis_calls":14,"redis_duration_s":0.0038970000000000003,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001138,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001386,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001373,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.04,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.04,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.180716,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000781385,"exclusive_lock_hold_duration_s":0.202023162,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.232794,"completed_at":"2026-09-01T10:34:17.284Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.054421,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1348gitlab # [ 186.653257] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.286Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"397d9627f5d098c32331bff3","created_at":"2026-09-01T10:34:15.906Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1af08a8d227ff0f4c1208a19be777b93","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.915Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-397d9627f5d098c32331bff3: start","job_status":"start","queue_duration_s":1.371735,"scheduling_latency_s":1.371765}1349gitlab # [ 186.654348] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.311Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"397d9627f5d098c32331bff3","created_at":"2026-09-01T10:34:15.906Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1af08a8d227ff0f4c1208a19be777b93","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:15.915Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-397d9627f5d098c32331bff3: done: 0.025013 sec","job_status":"done","queue_duration_s":1.371735,"scheduling_latency_s":1.371765,"redis_calls":10,"redis_duration_s":0.0025570000000000002,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000527,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00152,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00051,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015047,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000416254,"duration_s":0.025013,"completed_at":"2026-09-01T10:34:17.311Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000593,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1350gitlab # [ 186.696825] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.313Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"24b93de8a3b7869f13bda22a","created_at":"2026-09-01T10:34:15.925Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e05888d7bcb0178b8d871e78cb1274fa","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-01T10:34:15.939Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-24b93de8a3b7869f13bda22a: start","job_status":"start","queue_duration_s":1.374288,"scheduling_latency_s":1.374313}1351gitlab # [ 186.714814] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.345Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"24b93de8a3b7869f13bda22a","created_at":"2026-09-01T10:34:15.925Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e05888d7bcb0178b8d871e78cb1274fa","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-01T10:34:15.939Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-24b93de8a3b7869f13bda22a: done: 0.031336 sec","job_status":"done","queue_duration_s":1.374288,"scheduling_latency_s":1.374313,"redis_calls":10,"redis_duration_s":0.002457,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000409,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001533,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000515,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000394463,"duration_s":0.031336,"completed_at":"2026-09-01T10:34:17.345Z","load_balancing_strategy":"primary","db_duration_s":0.003486,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1352gitlab # [ 186.771167] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.347Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1663e9e44318fce0b88fe7d","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1663e9e44318fce0b88fe7d: start","job_status":"start","queue_duration_s":0.902638,"scheduling_latency_s":0.902664}1353gitlab # [ 186.782141] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.372Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1663e9e44318fce0b88fe7d","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1663e9e44318fce0b88fe7d: done: 0.025774 sec","job_status":"done","queue_duration_s":0.902638,"scheduling_latency_s":0.902664,"redis_calls":16,"redis_duration_s":0.0036750000000000003,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000402,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001089,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002184,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000437206,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025774,"completed_at":"2026-09-01T10:34:17.372Z","load_balancing_strategy":"primary","db_duration_s":0.000581,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1354gitlab # [ 186.894150] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.374Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b97effa2c014d258a1bc93d","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b97effa2c014d258a1bc93d: start","job_status":"start","queue_duration_s":0.929971,"scheduling_latency_s":0.929994}1355gitlab # [ 186.895364] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.391Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b97effa2c014d258a1bc93d","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b97effa2c014d258a1bc93d: done: 0.016887 sec","job_status":"done","queue_duration_s":0.929971,"scheduling_latency_s":0.929994,"redis_calls":13,"redis_duration_s":0.00279,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00085,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00194,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011201,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016887,"completed_at":"2026-09-01T10:34:17.391Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1356gitlab # [ 186.908867] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.393Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb61431ea75f25ee866d2a8a","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb61431ea75f25ee866d2a8a: start","job_status":"start","queue_duration_s":0.948504,"scheduling_latency_s":0.948528}1357gitlab # [ 186.926607] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb61431ea75f25ee866d2a8a","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb61431ea75f25ee866d2a8a: done: 0.017583 sec","job_status":"done","queue_duration_s":0.948504,"scheduling_latency_s":0.948528,"redis_calls":13,"redis_duration_s":0.002757,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001883,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017583,"completed_at":"2026-09-01T10:34:17.410Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1358gitlab # [ 187.002734] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.412Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bfc25a4af91380fca94cb0bb","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bfc25a4af91380fca94cb0bb: start","job_status":"start","queue_duration_s":0.96765,"scheduling_latency_s":0.967674}1359gitlab # [ 187.003991] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bfc25a4af91380fca94cb0bb","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bfc25a4af91380fca94cb0bb: done: 0.018084 sec","job_status":"done","queue_duration_s":0.96765,"scheduling_latency_s":0.967674,"redis_calls":13,"redis_duration_s":0.002809,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000858,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001951,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011575,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018084,"completed_at":"2026-09-01T10:34:17.430Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1360gitlab # [ 187.047781] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f4d9f4a16ac211b6abc4ed1e","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f4d9f4a16ac211b6abc4ed1e: start","job_status":"start","queue_duration_s":0.987433,"scheduling_latency_s":0.987457}1361gitlab # [ 187.065160] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.449Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f4d9f4a16ac211b6abc4ed1e","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f4d9f4a16ac211b6abc4ed1e: done: 0.017617 sec","job_status":"done","queue_duration_s":0.987433,"scheduling_latency_s":0.987457,"redis_calls":13,"redis_duration_s":0.002751,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000852,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001899,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017617,"completed_at":"2026-09-01T10:34:17.449Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1362gitlab # [ 187.137566] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.451Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e51b598c29b5d590a9601cf","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e51b598c29b5d590a9601cf: start","job_status":"start","queue_duration_s":1.006933,"scheduling_latency_s":1.006963}1363gitlab # [ 187.138753] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.469Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e51b598c29b5d590a9601cf","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e51b598c29b5d590a9601cf: done: 0.018294 sec","job_status":"done","queue_duration_s":1.006933,"scheduling_latency_s":1.006963,"redis_calls":13,"redis_duration_s":0.0030299999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002139,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018294,"completed_at":"2026-09-01T10:34:17.469Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1364gitlab # [ 187.185056] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.471Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9581a47417192981240af135","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9581a47417192981240af135: start","job_status":"start","queue_duration_s":1.026762,"scheduling_latency_s":1.026791}1365gitlab # [ 187.204484] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.488Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9581a47417192981240af135","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9581a47417192981240af135: done: 0.017311 sec","job_status":"done","queue_duration_s":1.026762,"scheduling_latency_s":1.026791,"redis_calls":13,"redis_duration_s":0.002682,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001811,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011541,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017311,"completed_at":"2026-09-01T10:34:17.488Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1366gitlab # [ 187.265281] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1214e2a6ce6a63e5866742b0","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1214e2a6ce6a63e5866742b0: start","job_status":"start","queue_duration_s":1.045865,"scheduling_latency_s":1.045892}1367gitlab # [ 187.274826] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.508Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1214e2a6ce6a63e5866742b0","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1214e2a6ce6a63e5866742b0: done: 0.018138 sec","job_status":"done","queue_duration_s":1.045865,"scheduling_latency_s":1.045892,"redis_calls":13,"redis_duration_s":0.002814,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00188,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018138,"completed_at":"2026-09-01T10:34:17.508Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1368gitlab # [ 187.325519] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.510Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2bee8cf737266099fe844183","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2bee8cf737266099fe844183: start","job_status":"start","queue_duration_s":1.065584,"scheduling_latency_s":1.06561}1369gitlab # [ 187.385062] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.531Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2bee8cf737266099fe844183","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2bee8cf737266099fe844183: done: 0.021468 sec","job_status":"done","queue_duration_s":1.065584,"scheduling_latency_s":1.06561,"redis_calls":13,"redis_duration_s":0.0030610000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002135,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013798,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021468,"completed_at":"2026-09-01T10:34:17.531Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1370gitlab # [ 187.394085] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.533Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8dc07e8aa4891a9d2ef33a75","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8dc07e8aa4891a9d2ef33a75: start","job_status":"start","queue_duration_s":1.089018,"scheduling_latency_s":1.08905}1371gitlab # [ 187.411477] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.554Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8dc07e8aa4891a9d2ef33a75","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8dc07e8aa4891a9d2ef33a75: done: 0.021497 sec","job_status":"done","queue_duration_s":1.089018,"scheduling_latency_s":1.08905,"redis_calls":13,"redis_duration_s":0.003503,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002449,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021497,"completed_at":"2026-09-01T10:34:17.554Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1372gitlab # [ 187.476057] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.556Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9db9d7f1ce6e6da9f3b3e1f7","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9db9d7f1ce6e6da9f3b3e1f7: start","job_status":"start","queue_duration_s":1.112489,"scheduling_latency_s":1.112517}1373gitlab # [ 187.477209] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.576Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9db9d7f1ce6e6da9f3b3e1f7","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9db9d7f1ce6e6da9f3b3e1f7: done: 0.019821 sec","job_status":"done","queue_duration_s":1.112489,"scheduling_latency_s":1.112517,"redis_calls":13,"redis_duration_s":0.0030889999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000973,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002116,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012906,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019821,"completed_at":"2026-09-01T10:34:17.576Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1374gitlab # [ 187.523551] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.578Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4124b29d52aa8257f31b6a58","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4124b29d52aa8257f31b6a58: start","job_status":"start","queue_duration_s":1.134098,"scheduling_latency_s":1.134137}1375gitlab # [ 187.540147] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.599Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4124b29d52aa8257f31b6a58","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4124b29d52aa8257f31b6a58: done: 0.020709 sec","job_status":"done","queue_duration_s":1.134098,"scheduling_latency_s":1.134137,"redis_calls":13,"redis_duration_s":0.00331,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001007,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002303,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013146,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020709,"completed_at":"2026-09-01T10:34:17.599Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1376gitlab # [ 187.615243] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a7c4aae28e646d301a34ba4","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a7c4aae28e646d301a34ba4: start","job_status":"start","queue_duration_s":1.156585,"scheduling_latency_s":1.156615}1377gitlab # [ 187.616467] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.619Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a7c4aae28e646d301a34ba4","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a7c4aae28e646d301a34ba4: done: 0.018776 sec","job_status":"done","queue_duration_s":1.156585,"scheduling_latency_s":1.156615,"redis_calls":13,"redis_duration_s":0.003066,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002195,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011507,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018776,"completed_at":"2026-09-01T10:34:17.619Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1378gitlab # [ 187.654201] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.621Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"456c711d0437d3517698f36e","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-456c711d0437d3517698f36e: start","job_status":"start","queue_duration_s":1.176772,"scheduling_latency_s":1.176799}1379gitlab # [ 187.669376] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.637Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"456c711d0437d3517698f36e","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-456c711d0437d3517698f36e: done: 0.016569 sec","job_status":"done","queue_duration_s":1.176772,"scheduling_latency_s":1.176799,"redis_calls":13,"redis_duration_s":0.002726,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000855,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001871,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016569,"completed_at":"2026-09-01T10:34:17.637Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1380gitlab # [ 187.726134] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.639Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a5bb5b71d81646c2d9f1c01","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a5bb5b71d81646c2d9f1c01: start","job_status":"start","queue_duration_s":1.194761,"scheduling_latency_s":1.194784}1381gitlab # [ 187.731615] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.660Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a5bb5b71d81646c2d9f1c01","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a5bb5b71d81646c2d9f1c01: done: 0.021529 sec","job_status":"done","queue_duration_s":1.194761,"scheduling_latency_s":1.194784,"redis_calls":13,"redis_duration_s":0.003359,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001138,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002221,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014283,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021529,"completed_at":"2026-09-01T10:34:17.660Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1382gitlab # [ 187.785095] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.663Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ca193549dd420d00640f301","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ca193549dd420d00640f301: start","job_status":"start","queue_duration_s":1.219191,"scheduling_latency_s":1.219231}1383gitlab # [ 187.802316] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.684Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ca193549dd420d00640f301","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ca193549dd420d00640f301: done: 0.020831 sec","job_status":"done","queue_duration_s":1.219191,"scheduling_latency_s":1.219231,"redis_calls":13,"redis_duration_s":0.0029389999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000864,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002075,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012498,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020831,"completed_at":"2026-09-01T10:34:17.684Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1384gitlab # [ 187.858784] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.686Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27ab6cff33ccb91550e36533","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27ab6cff33ccb91550e36533: start","job_status":"start","queue_duration_s":1.241772,"scheduling_latency_s":1.241801}1385gitlab # [ 187.928293] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27ab6cff33ccb91550e36533","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27ab6cff33ccb91550e36533: done: 0.018631 sec","job_status":"done","queue_duration_s":1.241772,"scheduling_latency_s":1.241801,"redis_calls":13,"redis_duration_s":0.002785,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001911,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018631,"completed_at":"2026-09-01T10:34:17.704Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1386gitlab # [ 187.991254] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.706Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56e00d458e857df77d20b53d","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56e00d458e857df77d20b53d: start","job_status":"start","queue_duration_s":1.262195,"scheduling_latency_s":1.262222}1387gitlab # [ 187.992662] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.727Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56e00d458e857df77d20b53d","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56e00d458e857df77d20b53d: done: 0.021237 sec","job_status":"done","queue_duration_s":1.262195,"scheduling_latency_s":1.262222,"redis_calls":13,"redis_duration_s":0.003315,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000986,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002329,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014207,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021237,"completed_at":"2026-09-01T10:34:17.727Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1388gitlab # [ 188.012096] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.729Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6dcc2ffb37e3d01f4969684b","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6dcc2ffb37e3d01f4969684b: start","job_status":"start","queue_duration_s":1.285431,"scheduling_latency_s":1.285457}1389gitlab # [ 188.029572] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.749Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6dcc2ffb37e3d01f4969684b","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6dcc2ffb37e3d01f4969684b: done: 0.019734 sec","job_status":"done","queue_duration_s":1.285431,"scheduling_latency_s":1.285457,"redis_calls":13,"redis_duration_s":0.0031799999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001024,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002156,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019734,"completed_at":"2026-09-01T10:34:17.749Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1390gitlab # [ 188.108818] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.751Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6c028c909e9c992324ad066","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6c028c909e9c992324ad066: start","job_status":"start","queue_duration_s":1.30707,"scheduling_latency_s":1.307102}1391gitlab # [ 188.110037] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.771Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6c028c909e9c992324ad066","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6c028c909e9c992324ad066: done: 0.019912 sec","job_status":"done","queue_duration_s":1.30707,"scheduling_latency_s":1.307102,"redis_calls":13,"redis_duration_s":0.0030629999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002172,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013311,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019912,"completed_at":"2026-09-01T10:34:17.771Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1392gitlab # [ 188.193884] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.773Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb3de50624abe3ab2e1a4cc3","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb3de50624abe3ab2e1a4cc3: start","job_status":"start","queue_duration_s":1.328513,"scheduling_latency_s":1.328539}1393gitlab # [ 188.194937] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.796Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb3de50624abe3ab2e1a4cc3","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb3de50624abe3ab2e1a4cc3: done: 0.0239 sec","job_status":"done","queue_duration_s":1.328513,"scheduling_latency_s":1.328539,"redis_calls":13,"redis_duration_s":0.003334,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001028,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002306,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015359,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0239,"completed_at":"2026-09-01T10:34:17.796Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1394gitlab # [ 188.215216] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.798Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"163df59d04e4fd83f849812c","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-163df59d04e4fd83f849812c: start","job_status":"start","queue_duration_s":1.354083,"scheduling_latency_s":1.354107}1395gitlab # [ 188.231112] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.823Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"163df59d04e4fd83f849812c","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-163df59d04e4fd83f849812c: done: 0.024652 sec","job_status":"done","queue_duration_s":1.354083,"scheduling_latency_s":1.354107,"redis_calls":13,"redis_duration_s":0.0054930000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001394,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004099,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014562,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024652,"completed_at":"2026-09-01T10:34:17.823Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1396gitlab # [ 188.280829] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.825Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"377c8ab8156101005059e46a","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-377c8ab8156101005059e46a: start","job_status":"start","queue_duration_s":1.380856,"scheduling_latency_s":1.380889}1397gitlab # [ 188.341253] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.847Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"377c8ab8156101005059e46a","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-377c8ab8156101005059e46a: done: 0.021733 sec","job_status":"done","queue_duration_s":1.380856,"scheduling_latency_s":1.380889,"redis_calls":13,"redis_duration_s":0.003242,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001017,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002225,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014392,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021733,"completed_at":"2026-09-01T10:34:17.846Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1398gitlab # [ 188.343281] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.849Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14a59fa4839c188087df3dae","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14a59fa4839c188087df3dae: start","job_status":"start","queue_duration_s":1.404792,"scheduling_latency_s":1.404865}1399gitlab # [ 188.359093] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.873Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14a59fa4839c188087df3dae","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14a59fa4839c188087df3dae: done: 0.024223 sec","job_status":"done","queue_duration_s":1.404792,"scheduling_latency_s":1.404865,"redis_calls":13,"redis_duration_s":0.003606,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001121,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002485,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015762,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024223,"completed_at":"2026-09-01T10:34:17.873Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1400gitlab # [ 188.457454] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.875Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"637cca61908f6324e0ee8c11","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-637cca61908f6324e0ee8c11: start","job_status":"start","queue_duration_s":1.430883,"scheduling_latency_s":1.430918}1401gitlab # [ 188.458453] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.895Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"637cca61908f6324e0ee8c11","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-637cca61908f6324e0ee8c11: done: 0.019714 sec","job_status":"done","queue_duration_s":1.430883,"scheduling_latency_s":1.430918,"redis_calls":13,"redis_duration_s":0.002979,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000824,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002155,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012301,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019714,"completed_at":"2026-09-01T10:34:17.894Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1402gitlab # [ 188.475077] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.896Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e56a171f3b1d2aa3a6229e8","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e56a171f3b1d2aa3a6229e8: start","job_status":"start","queue_duration_s":1.452076,"scheduling_latency_s":1.452101}1403gitlab # [ 188.490581] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.914Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e56a171f3b1d2aa3a6229e8","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e56a171f3b1d2aa3a6229e8: done: 0.018233 sec","job_status":"done","queue_duration_s":1.452076,"scheduling_latency_s":1.452101,"redis_calls":13,"redis_duration_s":0.0030440000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000984,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00206,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012167,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018233,"completed_at":"2026-09-01T10:34:17.914Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1404gitlab # [ 188.565110] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.916Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9917ab5bb763ab61cb97978f","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9917ab5bb763ab61cb97978f: start","job_status":"start","queue_duration_s":1.472107,"scheduling_latency_s":1.472143}1405gitlab # [ 188.566249] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.936Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9917ab5bb763ab61cb97978f","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9917ab5bb763ab61cb97978f: done: 0.019734 sec","job_status":"done","queue_duration_s":1.472107,"scheduling_latency_s":1.472143,"redis_calls":13,"redis_duration_s":0.0032459999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001061,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002185,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019734,"completed_at":"2026-09-01T10:34:17.936Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1406gitlab # [ 188.607408] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.938Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7fa7aac5c0468a388d5feae","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7fa7aac5c0468a388d5feae: start","job_status":"start","queue_duration_s":1.494126,"scheduling_latency_s":1.494161}1407gitlab # [ 188.623114] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.960Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7fa7aac5c0468a388d5feae","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7fa7aac5c0468a388d5feae: done: 0.021667 sec","job_status":"done","queue_duration_s":1.494126,"scheduling_latency_s":1.494161,"redis_calls":13,"redis_duration_s":0.002986,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000948,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021667,"completed_at":"2026-09-01T10:34:17.960Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1408gitlab # [ 188.687959] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.962Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4913e0694c7f947117f442e8","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4913e0694c7f947117f442e8: start","job_status":"start","queue_duration_s":1.517541,"scheduling_latency_s":1.517566}1409gitlab # [ 188.689096] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.983Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4913e0694c7f947117f442e8","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4913e0694c7f947117f442e8: done: 0.021882 sec","job_status":"done","queue_duration_s":1.517541,"scheduling_latency_s":1.517566,"redis_calls":13,"redis_duration_s":0.003244,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002261,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014556,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021882,"completed_at":"2026-09-01T10:34:17.983Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410gitlab # [ 188.739200] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:17.985Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"068db433237bd325722a1e7a","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-068db433237bd325722a1e7a: start","job_status":"start","queue_duration_s":1.541424,"scheduling_latency_s":1.541452}1411gitlab # [ 188.809294] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.008Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"068db433237bd325722a1e7a","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-068db433237bd325722a1e7a: done: 0.022354 sec","job_status":"done","queue_duration_s":1.541424,"scheduling_latency_s":1.541452,"redis_calls":13,"redis_duration_s":0.0033550000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001223,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002132,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022354,"completed_at":"2026-09-01T10:34:18.008Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412gitlab # [ 188.811238] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.010Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"82a29ed4127b6feb9bf654e0","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-82a29ed4127b6feb9bf654e0: start","job_status":"start","queue_duration_s":1.5659,"scheduling_latency_s":1.565927}1413gitlab # [ 188.824475] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.029Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"82a29ed4127b6feb9bf654e0","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-82a29ed4127b6feb9bf654e0: done: 0.019522 sec","job_status":"done","queue_duration_s":1.5659,"scheduling_latency_s":1.565927,"redis_calls":13,"redis_duration_s":0.003031,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000956,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002075,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012269,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019522,"completed_at":"2026-09-01T10:34:18.029Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1414gitlab # [ 188.879050] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.031Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f9545f577d7d85309fa0cfa2","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f9545f577d7d85309fa0cfa2: start","job_status":"start","queue_duration_s":1.586957,"scheduling_latency_s":1.586982}1415gitlab # [ 188.895448] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.048Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f9545f577d7d85309fa0cfa2","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f9545f577d7d85309fa0cfa2: done: 0.016728 sec","job_status":"done","queue_duration_s":1.586957,"scheduling_latency_s":1.586982,"redis_calls":13,"redis_duration_s":0.002615,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000866,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001749,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011129,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016728,"completed_at":"2026-09-01T10:34:18.048Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416gitlab # [ 188.950125] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.049Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d900afab94355c5dc5e8624","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d900afab94355c5dc5e8624: start","job_status":"start","queue_duration_s":1.60519,"scheduling_latency_s":1.605213}1417gitlab # [ 188.966135] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.071Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d900afab94355c5dc5e8624","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d900afab94355c5dc5e8624: done: 0.021829 sec","job_status":"done","queue_duration_s":1.60519,"scheduling_latency_s":1.605213,"redis_calls":13,"redis_duration_s":0.003292,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002239,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013934,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021829,"completed_at":"2026-09-01T10:34:18.071Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1418gitlab # [ 189.057246] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.073Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"240af30ec4a548601b8c74d9","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-240af30ec4a548601b8c74d9: start","job_status":"start","queue_duration_s":1.629061,"scheduling_latency_s":1.629092}1419gitlab # [ 189.058328] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.098Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"240af30ec4a548601b8c74d9","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-240af30ec4a548601b8c74d9: done: 0.024964 sec","job_status":"done","queue_duration_s":1.629061,"scheduling_latency_s":1.629092,"redis_calls":13,"redis_duration_s":0.0038510000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001139,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002712,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024964,"completed_at":"2026-09-01T10:34:18.098Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1420gitlab # [ 189.092701] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.100Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"882d0be2569b0b2dc37d5a3b","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-882d0be2569b0b2dc37d5a3b: start","job_status":"start","queue_duration_s":1.65623,"scheduling_latency_s":1.656267}1421gitlab # [ 189.113089] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"882d0be2569b0b2dc37d5a3b","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-882d0be2569b0b2dc37d5a3b: done: 0.025048 sec","job_status":"done","queue_duration_s":1.65623,"scheduling_latency_s":1.656267,"redis_calls":13,"redis_duration_s":0.004053,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001291,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002762,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025048,"completed_at":"2026-09-01T10:34:18.125Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1422gitlab # [ 189.189466] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.127Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57942478134486fea909ba22","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57942478134486fea909ba22: start","job_status":"start","queue_duration_s":1.68296,"scheduling_latency_s":1.682986}1423gitlab # [ 189.190807] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.146Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57942478134486fea909ba22","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57942478134486fea909ba22: done: 0.018864 sec","job_status":"done","queue_duration_s":1.68296,"scheduling_latency_s":1.682986,"redis_calls":13,"redis_duration_s":0.002744,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000833,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001911,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012511,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018864,"completed_at":"2026-09-01T10:34:18.146Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1424gitlab # [ 189.235686] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.147Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6eee6fbf4409003ff4dccb2d","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6eee6fbf4409003ff4dccb2d: start","job_status":"start","queue_duration_s":1.703456,"scheduling_latency_s":1.703482}1425gitlab # [ 189.252681] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6eee6fbf4409003ff4dccb2d","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6eee6fbf4409003ff4dccb2d: done: 0.020589 sec","job_status":"done","queue_duration_s":1.703456,"scheduling_latency_s":1.703482,"redis_calls":13,"redis_duration_s":0.0034279999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001152,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002276,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01416,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020589,"completed_at":"2026-09-01T10:34:18.168Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1426gitlab # [ 189.308218] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.170Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c67223eac1a6052f96c027e4","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c67223eac1a6052f96c027e4: start","job_status":"start","queue_duration_s":1.725776,"scheduling_latency_s":1.725803}1427gitlab # [ 189.326435] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.187Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c67223eac1a6052f96c027e4","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c67223eac1a6052f96c027e4: done: 0.017623 sec","job_status":"done","queue_duration_s":1.725776,"scheduling_latency_s":1.725803,"redis_calls":13,"redis_duration_s":0.002776,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000861,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001915,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011486,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017623,"completed_at":"2026-09-01T10:34:18.187Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1428gitlab # [ 189.382168] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.189Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"478edf064bacf81b307bccb1","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-478edf064bacf81b307bccb1: start","job_status":"start","queue_duration_s":1.745211,"scheduling_latency_s":1.745241}1429gitlab # [ 189.435851] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.209Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"478edf064bacf81b307bccb1","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-478edf064bacf81b307bccb1: done: 0.019559 sec","job_status":"done","queue_duration_s":1.745211,"scheduling_latency_s":1.745241,"redis_calls":13,"redis_duration_s":0.0029230000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01307,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019559,"completed_at":"2026-09-01T10:34:18.209Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1430gitlab # [ 189.446081] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.211Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff7e912a8a45cb3307ea77f7","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff7e912a8a45cb3307ea77f7: start","job_status":"start","queue_duration_s":1.766723,"scheduling_latency_s":1.766753}1431gitlab # [ 189.462208] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.231Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff7e912a8a45cb3307ea77f7","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff7e912a8a45cb3307ea77f7: done: 0.020017 sec","job_status":"done","queue_duration_s":1.766723,"scheduling_latency_s":1.766753,"redis_calls":13,"redis_duration_s":0.003196,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001131,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002065,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013076,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020017,"completed_at":"2026-09-01T10:34:18.231Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1432gitlab # [ 189.552467] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.233Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c127750cce121fdee262854","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c127750cce121fdee262854: start","job_status":"start","queue_duration_s":1.788866,"scheduling_latency_s":1.788893}1433gitlab # [ 189.553929] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.252Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c127750cce121fdee262854","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c127750cce121fdee262854: done: 0.019663 sec","job_status":"done","queue_duration_s":1.788866,"scheduling_latency_s":1.788893,"redis_calls":13,"redis_duration_s":0.002908,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000949,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001959,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019663,"completed_at":"2026-09-01T10:34:18.252Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1434gitlab # [ 189.581163] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.255Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38ca969803c2497c60108f2b","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38ca969803c2497c60108f2b: start","job_status":"start","queue_duration_s":1.810506,"scheduling_latency_s":1.810534}1435gitlab # [ 189.597805] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.274Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38ca969803c2497c60108f2b","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38ca969803c2497c60108f2b: done: 0.019697 sec","job_status":"done","queue_duration_s":1.810506,"scheduling_latency_s":1.810534,"redis_calls":13,"redis_duration_s":0.003291,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001085,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002206,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012817,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019697,"completed_at":"2026-09-01T10:34:18.274Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1436gitlab # [ 189.675262] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.276Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de477f281a568de9b7b4e80d","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de477f281a568de9b7b4e80d: start","job_status":"start","queue_duration_s":1.832034,"scheduling_latency_s":1.832058}1437gitlab # [ 189.676764] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.296Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de477f281a568de9b7b4e80d","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de477f281a568de9b7b4e80d: done: 0.020038 sec","job_status":"done","queue_duration_s":1.832034,"scheduling_latency_s":1.832058,"redis_calls":13,"redis_duration_s":0.003127,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000878,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002249,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013688,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020038,"completed_at":"2026-09-01T10:34:18.296Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1438gitlab # [ 189.719943] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.298Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dfe9718210fd644aaabe92fb","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dfe9718210fd644aaabe92fb: start","job_status":"start","queue_duration_s":1.853647,"scheduling_latency_s":1.853672}1439gitlab # [ 189.738160] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dfe9718210fd644aaabe92fb","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dfe9718210fd644aaabe92fb: done: 0.022413 sec","job_status":"done","queue_duration_s":1.853647,"scheduling_latency_s":1.853672,"redis_calls":13,"redis_duration_s":0.003543,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00249,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01519,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022413,"completed_at":"2026-09-01T10:34:18.320Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1440gitlab # [ 189.793905] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.322Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"243fb2ab66946aa02e17a20c","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-243fb2ab66946aa02e17a20c: start","job_status":"start","queue_duration_s":1.878357,"scheduling_latency_s":1.878383}1441gitlab # [ 189.811044] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.343Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"243fb2ab66946aa02e17a20c","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-243fb2ab66946aa02e17a20c: done: 0.020403 sec","job_status":"done","queue_duration_s":1.878357,"scheduling_latency_s":1.878383,"redis_calls":13,"redis_duration_s":0.0031019999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000978,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002124,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020403,"completed_at":"2026-09-01T10:34:18.343Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1442gitlab # [ 189.892303] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.344Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b722162f51d7f54beca50c71","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b722162f51d7f54beca50c71: start","job_status":"start","queue_duration_s":1.900359,"scheduling_latency_s":1.900384}1443gitlab # [ 189.893945] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.362Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b722162f51d7f54beca50c71","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b722162f51d7f54beca50c71: done: 0.018119 sec","job_status":"done","queue_duration_s":1.900359,"scheduling_latency_s":1.900384,"redis_calls":13,"redis_duration_s":0.0028179999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000855,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011915,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018119,"completed_at":"2026-09-01T10:34:18.362Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1444gitlab # [ 189.936079] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"759cf50310f95f2896103ecb","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-759cf50310f95f2896103ecb: start","job_status":"start","queue_duration_s":1.920148,"scheduling_latency_s":1.920173}1445gitlab # [ 189.952612] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.384Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"759cf50310f95f2896103ecb","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-759cf50310f95f2896103ecb: done: 0.01961 sec","job_status":"done","queue_duration_s":1.920148,"scheduling_latency_s":1.920173,"redis_calls":13,"redis_duration_s":0.0029909999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000852,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002139,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013058,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01961,"completed_at":"2026-09-01T10:34:18.384Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1446gitlab # [ 190.048968] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.386Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ad2c9a40544d8e10e48eb72","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ad2c9a40544d8e10e48eb72: start","job_status":"start","queue_duration_s":1.941722,"scheduling_latency_s":1.941752}1447gitlab # [ 190.050142] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.407Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ad2c9a40544d8e10e48eb72","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ad2c9a40544d8e10e48eb72: done: 0.021362 sec","job_status":"done","queue_duration_s":1.941722,"scheduling_latency_s":1.941752,"redis_calls":13,"redis_duration_s":0.003184,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001033,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002151,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013469,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021362,"completed_at":"2026-09-01T10:34:18.407Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1448gitlab # [ 190.071485] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.409Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1847c79fbd9b30bf319b15c9","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1847c79fbd9b30bf319b15c9: start","job_status":"start","queue_duration_s":1.965028,"scheduling_latency_s":1.965059}1449gitlab # [ 190.088186] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1847c79fbd9b30bf319b15c9","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1847c79fbd9b30bf319b15c9: done: 0.019948 sec","job_status":"done","queue_duration_s":1.965028,"scheduling_latency_s":1.965059,"redis_calls":13,"redis_duration_s":0.0033870000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002442,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019948,"completed_at":"2026-09-01T10:34:18.429Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1450gitlab # [ 190.147350] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0decdcd8ae53928a8142fb1c","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0decdcd8ae53928a8142fb1c: start","job_status":"start","queue_duration_s":1.986653,"scheduling_latency_s":1.986678}1451gitlab # [ 190.164087] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.448Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0decdcd8ae53928a8142fb1c","created_at":"2026-09-01T10:34:16.365Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-62fb564ad2f844a3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"a540adb6be0e9464179a7f827a59e3bd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.444Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0decdcd8ae53928a8142fb1c: done: 0.017879 sec","job_status":"done","queue_duration_s":1.986653,"scheduling_latency_s":1.986678,"redis_calls":13,"redis_duration_s":0.002811,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00189,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012019,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017879,"completed_at":"2026-09-01T10:34:18.448Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1452gitlab # [ 190.220086] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.450Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"091e3155f343258845bd4813","created_at":"2026-09-01T10:34:16.862Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-6229693a30274481","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"26dff5e947c682e66168ce55b58c8c89","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.874Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-091e3155f343258845bd4813: start","job_status":"start","queue_duration_s":1.576461,"scheduling_latency_s":1.576487}1453gitlab # [ 190.229776] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.469Z","message":"No existing merge request to be cleaned up."}1454gitlab # [ 190.232045] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.483Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"091e3155f343258845bd4813","created_at":"2026-09-01T10:34:16.862Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-6229693a30274481","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"26dff5e947c682e66168ce55b58c8c89","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.874Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-091e3155f343258845bd4813: done: 0.032992 sec","job_status":"done","queue_duration_s":1.576461,"scheduling_latency_s":1.576487,"redis_calls":13,"redis_duration_s":0.0032649999999999997,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000477,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001246,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001542,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020718,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000428547,"duration_s":0.032992,"completed_at":"2026-09-01T10:34:18.483Z","load_balancing_strategy":"primary","db_duration_s":0.004617,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455gitlab # [ 190.295201] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.485Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3287124d2669088f54b33699","created_at":"2026-09-01T10:34:16.862Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-6229693a30274481","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"26dff5e947c682e66168ce55b58c8c89","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.874Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3287124d2669088f54b33699: start","job_status":"start","queue_duration_s":1.611096,"scheduling_latency_s":1.611122}1456gitlab # [ 190.301326] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.497Z","message":"No existing merge request to be cleaned up."}1457gitlab # [ 190.304223] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.505Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3287124d2669088f54b33699","created_at":"2026-09-01T10:34:16.862Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-6229693a30274481","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"26dff5e947c682e66168ce55b58c8c89","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.874Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3287124d2669088f54b33699: done: 0.019827 sec","job_status":"done","queue_duration_s":1.611096,"scheduling_latency_s":1.611122,"redis_calls":10,"redis_duration_s":0.002357,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000904,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001453,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019827,"completed_at":"2026-09-01T10:34:18.505Z","load_balancing_strategy":"primary","db_duration_s":0.001868,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1458gitlab # [ 190.359148] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.506Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"57440ab4e7befef7ecf13888","created_at":"2026-09-01T10:34:16.862Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-6229693a30274481","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"26dff5e947c682e66168ce55b58c8c89","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.874Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-57440ab4e7befef7ecf13888: start","job_status":"start","queue_duration_s":1.632487,"scheduling_latency_s":1.632514}1459gitlab # [ 190.418073] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.518Z","message":"No existing merge request to be cleaned up."}1460gitlab # [ 190.419353] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.526Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"57440ab4e7befef7ecf13888","created_at":"2026-09-01T10:34:16.862Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-6229693a30274481","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"26dff5e947c682e66168ce55b58c8c89","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.874Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-57440ab4e7befef7ecf13888: done: 0.019837 sec","job_status":"done","queue_duration_s":1.632487,"scheduling_latency_s":1.632514,"redis_calls":10,"redis_duration_s":0.0025570000000000002,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001265,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001292,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019837,"completed_at":"2026-09-01T10:34:18.526Z","load_balancing_strategy":"primary","db_duration_s":0.001691,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461gitlab # [ 190.427793] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.528Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c65f596e6705fb869f8a34b8","created_at":"2026-09-01T10:34:16.862Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-6229693a30274481","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"26dff5e947c682e66168ce55b58c8c89","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.874Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c65f596e6705fb869f8a34b8: start","job_status":"start","queue_duration_s":1.653861,"scheduling_latency_s":1.653886}1462gitlab # [ 190.443753] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.540Z","message":"No existing merge request to be cleaned up."}1463gitlab # [ 190.445761] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:34:18.549Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c65f596e6705fb869f8a34b8","created_at":"2026-09-01T10:34:16.862Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-6229693a30274481","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"26dff5e947c682e66168ce55b58c8c89","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:16.874Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c65f596e6705fb869f8a34b8: done: 0.021623 sec","job_status":"done","queue_duration_s":1.653861,"scheduling_latency_s":1.653886,"redis_calls":10,"redis_duration_s":0.002519,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001065,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001454,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014596,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021623,"completed_at":"2026-09-01T10:34:18.549Z","load_balancing_strategy":"primary","db_duration_s":0.001983,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1464gitlab # [ 190.788758] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 4253926097 wd_nsec: 42539245521465gitlab # [ 219.380867] setup-gitlab-start[2600]: W, [2026-09-01T10:34:50.538441 #2600] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1466gitlab # [ 230.499358] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:01.656Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"3a4f2852e7f71dd8bd962808","created_at":"2026-09-01T10:35:01.629Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"33b4b8616d0177e22430563dd4bad0e7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-01T10:35:01.646Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-3a4f2852e7f71dd8bd962808: start","job_status":"start","queue_duration_s":0.009587,"scheduling_latency_s":0.009646}1467gitlab # [ 230.586819] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:01.744Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1468gitlab # [ 230.638169] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:01.792Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"3a4f2852e7f71dd8bd962808","created_at":"2026-09-01T10:35:01.629Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"33b4b8616d0177e22430563dd4bad0e7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-01T10:35:01.646Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-3a4f2852e7f71dd8bd962808: done: 0.136363 sec","job_status":"done","queue_duration_s":0.009587,"scheduling_latency_s":0.009646,"redis_calls":8,"redis_duration_s":0.030195,"redis_read_bytes":9,"redis_write_bytes":810,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.026192,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":552,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004003,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019853,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.136363,"completed_at":"2026-09-01T10:35:01.792Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1469gitlab # [ 230.701391] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:01.833Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"a0e23b3e897190d36d8d0cf3","created_at":"2026-09-01T10:35:01.711Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d83d28f195c14e3ede6a687e36d8a6f3","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:01.821Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-a0e23b3e897190d36d8d0cf3: start","job_status":"start","queue_duration_s":0.011551,"scheduling_latency_s":0.011593}1470gitlab # [ 232.712939] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:03.869Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"a0e23b3e897190d36d8d0cf3","created_at":"2026-09-01T10:35:01.711Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d83d28f195c14e3ede6a687e36d8a6f3","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:01.821Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-a0e23b3e897190d36d8d0cf3: done: 2.036371 sec","job_status":"done","queue_duration_s":0.011551,"scheduling_latency_s":0.011593,"redis_calls":562,"redis_duration_s":1.3721219999999998,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.010962,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":1.36116,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.577834,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":2.036371,"completed_at":"2026-09-01T10:35:03.869Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1471gitlab # [ 232.720160] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:03.877Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"b13898b829af8b3cae009c31","created_at":"2026-09-01T10:35:01.832Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67c54ce6f9ae1e446d5deb02780659d7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:01.843Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-b13898b829af8b3cae009c31: start","job_status":"start","queue_duration_s":2.03427,"scheduling_latency_s":2.034323}1472gitlab # [ 232.788689] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:03.946Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"b13898b829af8b3cae009c31","created_at":"2026-09-01T10:35:01.832Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67c54ce6f9ae1e446d5deb02780659d7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:01.843Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-b13898b829af8b3cae009c31: done: 0.068826 sec","job_status":"done","queue_duration_s":2.03427,"scheduling_latency_s":2.034323,"redis_calls":10,"redis_duration_s":0.026181,"redis_read_bytes":135,"redis_write_bytes":1121,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.00183,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":49,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.009914,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.014437,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019364,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001160203,"exclusive_lock_hold_duration_s":0.009966351,"duration_s":0.068826,"completed_at":"2026-09-01T10:35:03.946Z","load_balancing_strategy":"primary","db_duration_s":0.002461,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1473gitlab # [ 232.798091] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:03.955Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"ad20bc5ab26c718298d9ef35","created_at":"2026-09-01T10:35:01.856Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"45b853300495760580a15bc718c7867a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:01.870Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-ad20bc5ab26c718298d9ef35: start","job_status":"start","queue_duration_s":2.084926,"scheduling_latency_s":2.085017}1474gitlab # [ 233.241530] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:04.396Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"ad20bc5ab26c718298d9ef35","created_at":"2026-09-01T10:35:01.856Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"45b853300495760580a15bc718c7867a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:01.870Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-ad20bc5ab26c718298d9ef35: done: 0.442104 sec","job_status":"done","queue_duration_s":2.084926,"scheduling_latency_s":2.085017,"redis_calls":12,"redis_duration_s":0.040584999999999996,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004485,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008258,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.027842,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.022,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.257433,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.003828699,"exclusive_lock_hold_duration_s":0.315400421,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.442104,"completed_at":"2026-09-01T10:35:04.396Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.063217,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null}1475gitlab # [ 233.336514] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:04.442Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"bcbb5c144b76af5dd3041734","created_at":"2026-09-01T10:35:01.885Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:01.897Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-bcbb5c144b76af5dd3041734: start","job_status":"start","queue_duration_s":2.54536,"scheduling_latency_s":2.545404}1476gitlab # [ 233.825378] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:04.982Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"bcbb5c144b76af5dd3041734","created_at":"2026-09-01T10:35:01.885Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:01.897Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-bcbb5c144b76af5dd3041734: done: 0.540174 sec","job_status":"done","queue_duration_s":2.54536,"scheduling_latency_s":2.545404,"redis_calls":62,"redis_duration_s":0.262255,"redis_read_bytes":364,"redis_write_bytes":41241,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00327,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":132,"redis_queues_calls":2,"redis_queues_duration_s":0.002783,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.213311,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.042891,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.165522,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.540174,"completed_at":"2026-09-01T10:35:04.982Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1477gitlab # [ 233.903727] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:05.061Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"5e731d724419853507432554","created_at":"2026-09-01T10:35:01.916Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2ec27c9ed952e350e01605ad8a600c39","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:01.920Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-5e731d724419853507432554: start","job_status":"start","queue_duration_s":3.140338,"scheduling_latency_s":3.140381}1478gitlab # [ 234.003294] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:05.160Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"5e731d724419853507432554","created_at":"2026-09-01T10:35:01.916Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2ec27c9ed952e350e01605ad8a600c39","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:01.920Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-5e731d724419853507432554: done: 0.099792 sec","job_status":"done","queue_duration_s":3.140338,"scheduling_latency_s":3.140381,"redis_calls":13,"redis_duration_s":0.029977,"redis_read_bytes":773,"redis_write_bytes":1102,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.014074,"redis_feature_flag_read_bytes":768,"redis_feature_flag_write_bytes":391,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.01255,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003353,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02412,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.099792,"completed_at":"2026-09-01T10:35:05.160Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014557,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1479gitlab # [ 234.074248] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:05.231Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"6ebb94bb2f7576d35dc7099f","created_at":"2026-09-01T10:35:01.940Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8daedecd3d8f9a980fccc47057e65b5f","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:01.948Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-6ebb94bb2f7576d35dc7099f: start","job_status":"start","queue_duration_s":3.282721,"scheduling_latency_s":3.282766}1480gitlab # [ 234.446410] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:05.603Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"6ebb94bb2f7576d35dc7099f","created_at":"2026-09-01T10:35:01.940Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8daedecd3d8f9a980fccc47057e65b5f","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:01.948Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-6ebb94bb2f7576d35dc7099f: done: 0.372286 sec","job_status":"done","queue_duration_s":3.282721,"scheduling_latency_s":3.282766,"redis_calls":7,"redis_duration_s":0.029275999999999996,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.015846,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.01343,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025079,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.372286,"completed_at":"2026-09-01T10:35:05.603Z","load_balancing_strategy":"primary","db_duration_s":0.023506,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1481gitlab # [ 234.461588] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:05.617Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"0b13521d4df076860e13d939","created_at":"2026-09-01T10:35:01.960Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ae058aba7e84928cb0cd2bdfd17f7cb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:01.966Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-0b13521d4df076860e13d939: start","job_status":"start","queue_duration_s":3.650644,"scheduling_latency_s":3.650689}1482gitlab # [ 234.593372] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:05.638Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1483gitlab # [ 234.593610] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:05.654Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"0b13521d4df076860e13d939","created_at":"2026-09-01T10:35:01.960Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ae058aba7e84928cb0cd2bdfd17f7cb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:01.966Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-0b13521d4df076860e13d939: done: 0.037514 sec","job_status":"done","queue_duration_s":3.650644,"scheduling_latency_s":3.650689,"redis_calls":7,"redis_duration_s":0.002604,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001842,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000762,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037514,"completed_at":"2026-09-01T10:35:05.654Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1484gitlab # [ 234.672933] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:05.657Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"7d7b06e588e6b4e4c78f2e26","created_at":"2026-09-01T10:35:01.991Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b767543fc47a14bae971c7a534fa85bb","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.004Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-7d7b06e588e6b4e4c78f2e26: start","job_status":"start","queue_duration_s":3.652831,"scheduling_latency_s":3.652865}1485gitlab # [ 234.676339] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:05.700Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"7d7b06e588e6b4e4c78f2e26","created_at":"2026-09-01T10:35:01.991Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b767543fc47a14bae971c7a534fa85bb","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.004Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-7d7b06e588e6b4e4c78f2e26: done: 0.043994 sec","job_status":"done","queue_duration_s":3.652831,"scheduling_latency_s":3.652865,"redis_calls":11,"redis_duration_s":0.009874,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001329,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004704,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.002487,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001354,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024286,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000745347,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.043994,"completed_at":"2026-09-01T10:35:05.700Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001375,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1486gitlab # [ 234.788119] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:05.705Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"8b8aca28a106bd3c170cc813","created_at":"2026-09-01T10:35:02.023Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0f4d37f087c84de48d7d5e8aa503da11","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.028Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-8b8aca28a106bd3c170cc813: start","job_status":"start","queue_duration_s":3.677036,"scheduling_latency_s":3.677083}1487gitlab # [ 234.798502] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:05.738Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"8b8aca28a106bd3c170cc813","created_at":"2026-09-01T10:35:02.023Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0f4d37f087c84de48d7d5e8aa503da11","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.028Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-8b8aca28a106bd3c170cc813: done: 0.033809 sec","job_status":"done","queue_duration_s":3.677036,"scheduling_latency_s":3.677083,"redis_calls":9,"redis_duration_s":0.005514,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002819,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002695,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019533,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001161042,"exclusive_lock_hold_duration_s":0.005010972,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.033809,"completed_at":"2026-09-01T10:35:05.738Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001923,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1488gitlab # [ 234.896757] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:05.742Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"2f537eef9575f1c97b7a1740","created_at":"2026-09-01T10:35:02.082Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"647d94bb92049433458a77c326fe0822","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.088Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-2f537eef9575f1c97b7a1740: start","job_status":"start","queue_duration_s":3.654205,"scheduling_latency_s":3.654255}1489gitlab # [ 234.898146] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:05.764Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1490gitlab # [ 234.899300] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:05.783Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"2f537eef9575f1c97b7a1740","created_at":"2026-09-01T10:35:02.082Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"647d94bb92049433458a77c326fe0822","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.088Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-2f537eef9575f1c97b7a1740: done: 0.041083 sec","job_status":"done","queue_duration_s":3.654205,"scheduling_latency_s":3.654255,"redis_calls":7,"redis_duration_s":0.010818000000000001,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00982,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017383,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041083,"completed_at":"2026-09-01T10:35:05.783Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1491gitlab # [ 234.986265] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:05.787Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"d6827f8a5b83116764830299","created_at":"2026-09-01T10:35:02.103Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba522c2d3ec18b2452ea625abe2bfb30","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.111Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-d6827f8a5b83116764830299: start","job_status":"start","queue_duration_s":3.67662,"scheduling_latency_s":3.676665}1492gitlab # [ 235.000528] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:05.820Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"d6827f8a5b83116764830299","created_at":"2026-09-01T10:35:02.103Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba522c2d3ec18b2452ea625abe2bfb30","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.111Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-d6827f8a5b83116764830299: done: 0.032953 sec","job_status":"done","queue_duration_s":3.67662,"scheduling_latency_s":3.676665,"redis_calls":7,"redis_duration_s":0.006453,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003621,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002832,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017449,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032953,"completed_at":"2026-09-01T10:35:05.820Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001307,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1493gitlab # [ 235.082241] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:05.830Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"453360fcd83fa17bce8e7f79","created_at":"2026-09-01T10:35:02.133Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c534b1412b2feec14db58fb04df6edf1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.138Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-453360fcd83fa17bce8e7f79: start","job_status":"start","queue_duration_s":3.692356,"scheduling_latency_s":3.692411}1494gitlab # [ 235.083633] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:06.115Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"453360fcd83fa17bce8e7f79","created_at":"2026-09-01T10:35:02.133Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c534b1412b2feec14db58fb04df6edf1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.138Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-453360fcd83fa17bce8e7f79: done: 0.286057 sec","job_status":"done","queue_duration_s":3.692356,"scheduling_latency_s":3.692411,"redis_calls":9,"redis_duration_s":0.012598,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005494,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.007104,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.109,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.109,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.002117588,"exclusive_lock_hold_duration_s":0.187588367,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.286057,"completed_at":"2026-09-01T10:35:06.115Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.163753,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1495gitlab # [ 235.191296] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:06.120Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"ba7833b9b0b994d2166adb3b","created_at":"2026-09-01T10:35:02.152Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3df57e7460e46f76d16a1a6e0bb017ea","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.156Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-ba7833b9b0b994d2166adb3b: start","job_status":"start","queue_duration_s":3.963648,"scheduling_latency_s":3.963694}1496gitlab # [ 235.280585] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:06.437Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"ba7833b9b0b994d2166adb3b","created_at":"2026-09-01T10:35:02.152Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3df57e7460e46f76d16a1a6e0bb017ea","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.156Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-ba7833b9b0b994d2166adb3b: done: 0.316723 sec","job_status":"done","queue_duration_s":3.963648,"scheduling_latency_s":3.963694,"redis_calls":14,"redis_duration_s":0.046417,"redis_read_bytes":16,"redis_write_bytes":4026,"redis_queues_calls":2,"redis_queues_duration_s":0.001201,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.031059,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.014157,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.077,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.077,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.095766,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.316723,"completed_at":"2026-09-01T10:35:06.437Z","load_balancing_strategy":"primary","db_duration_s":0.07674,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1497gitlab # [ 235.376642] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:06.534Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"574a27bef65bfc1f22888eda","created_at":"2026-09-01T10:35:02.170Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21aea1dc06d12aec597fadbf484ac16c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.177Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-574a27bef65bfc1f22888eda: start","job_status":"start","queue_duration_s":4.356368,"scheduling_latency_s":4.356414}1498gitlab # [ 235.483832] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:06.641Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1499gitlab # [ 235.525702] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:06.682Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"574a27bef65bfc1f22888eda","created_at":"2026-09-01T10:35:02.170Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21aea1dc06d12aec597fadbf484ac16c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.177Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-574a27bef65bfc1f22888eda: done: 0.148709 sec","job_status":"done","queue_duration_s":4.356368,"scheduling_latency_s":4.356414,"redis_calls":7,"redis_duration_s":0.042834,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.022962,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.019872,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016221,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.148709,"completed_at":"2026-09-01T10:35:06.682Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1500gitlab # [ 235.668859] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:06.687Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"f37a03ca300972c8a019f607","created_at":"2026-09-01T10:35:02.206Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"418501812390b9168514053d431d17b2","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.220Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-f37a03ca300972c8a019f607: start","job_status":"start","queue_duration_s":4.467095,"scheduling_latency_s":4.467139}1501gitlab # [ 235.729825] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:06.883Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"f37a03ca300972c8a019f607","created_at":"2026-09-01T10:35:02.206Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"418501812390b9168514053d431d17b2","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.220Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-f37a03ca300972c8a019f607: done: 0.196131 sec","job_status":"done","queue_duration_s":4.467095,"scheduling_latency_s":4.467139,"redis_calls":10,"redis_duration_s":0.08524400000000001,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.067626,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.017618,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.040264615,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.196131,"completed_at":"2026-09-01T10:35:06.883Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026967,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1502gitlab # [ 235.834074] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:06.895Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"69326ea868a2fc23f4e2a78c","created_at":"2026-09-01T10:35:02.274Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2b3bbadfdaf6e7a2cbcd9ec82dc70161","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.278Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-69326ea868a2fc23f4e2a78c: start","job_status":"start","queue_duration_s":4.616984,"scheduling_latency_s":4.617029}1503gitlab # [ 236.211378] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:07.368Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"69326ea868a2fc23f4e2a78c","created_at":"2026-09-01T10:35:02.274Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2b3bbadfdaf6e7a2cbcd9ec82dc70161","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.278Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-69326ea868a2fc23f4e2a78c: done: 0.473326 sec","job_status":"done","queue_duration_s":4.616984,"scheduling_latency_s":4.617029,"redis_calls":9,"redis_duration_s":0.29857300000000003,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.284287,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.014286,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.037,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.003262705,"exclusive_lock_hold_duration_s":0.047476983,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.473326,"completed_at":"2026-09-01T10:35:07.368Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.054197,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1504gitlab # [ 236.297627] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:07.381Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"314e3d01d9ae35b466083ee9","created_at":"2026-09-01T10:35:02.307Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8bde54c41bb8d7ecd9826b40ac6eebbc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.311Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-314e3d01d9ae35b466083ee9: start","job_status":"start","queue_duration_s":5.07087,"scheduling_latency_s":5.070915}1505gitlab # [ 236.624601] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:07.780Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"314e3d01d9ae35b466083ee9","created_at":"2026-09-01T10:35:02.307Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8bde54c41bb8d7ecd9826b40ac6eebbc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.311Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-314e3d01d9ae35b466083ee9: done: 0.398768 sec","job_status":"done","queue_duration_s":5.07087,"scheduling_latency_s":5.070915,"redis_calls":8,"redis_duration_s":0.044272,"redis_read_bytes":133,"redis_write_bytes":846,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.00072,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.042779,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000773,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016835,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.398768,"completed_at":"2026-09-01T10:35:07.780Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1506gitlab # [ 236.747155] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:07.903Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"c1c69c9748373b3bdcd56814","created_at":"2026-09-01T10:35:02.342Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f519d78647f4c3aa542e43cd9c96395","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.349Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-c1c69c9748373b3bdcd56814: start","job_status":"start","queue_duration_s":5.553942,"scheduling_latency_s":5.553998}1507gitlab # [ 236.798809] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:07.956Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1508gitlab # [ 236.827478] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:07.984Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"c1c69c9748373b3bdcd56814","created_at":"2026-09-01T10:35:02.342Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f519d78647f4c3aa542e43cd9c96395","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.349Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-c1c69c9748373b3bdcd56814: done: 0.201759 sec","job_status":"done","queue_duration_s":5.553942,"scheduling_latency_s":5.553998,"redis_calls":7,"redis_duration_s":0.043303999999999995,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.010494,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.03281,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015172,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.201759,"completed_at":"2026-09-01T10:35:07.984Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1509gitlab # [ 236.905542] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:08.062Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"853ebec9bd8736fbdeb11231","created_at":"2026-09-01T10:35:02.376Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3b238a585947e5e95cd9c65e6dad5911","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-01T10:35:02.380Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-853ebec9bd8736fbdeb11231: start","job_status":"start","queue_duration_s":5.681727,"scheduling_latency_s":5.681775}1510gitlab # [ 236.937352] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:08.095Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1511gitlab # [ 236.960063] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:08.117Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"853ebec9bd8736fbdeb11231","created_at":"2026-09-01T10:35:02.376Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3b238a585947e5e95cd9c65e6dad5911","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-01T10:35:02.380Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-853ebec9bd8736fbdeb11231: done: 0.054656 sec","job_status":"done","queue_duration_s":5.681727,"scheduling_latency_s":5.681775,"redis_calls":7,"redis_duration_s":0.0072900000000000005,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006525,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000765,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016708,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.054656,"completed_at":"2026-09-01T10:35:08.117Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1512gitlab # [ 236.995935] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:08.125Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"8a898188e98a25af5443950f","created_at":"2026-09-01T10:35:02.401Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3ccf36aea3d034b999f39092b05402f","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.417Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-8a898188e98a25af5443950f: start","job_status":"start","queue_duration_s":5.707308,"scheduling_latency_s":5.707353}1513gitlab # [ 237.037303] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:08.194Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"8a898188e98a25af5443950f","created_at":"2026-09-01T10:35:02.401Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3ccf36aea3d034b999f39092b05402f","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.417Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-8a898188e98a25af5443950f: done: 0.069768 sec","job_status":"done","queue_duration_s":5.707308,"scheduling_latency_s":5.707353,"redis_calls":10,"redis_duration_s":0.011039,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000725,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009638,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000676,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026847,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000535822,"duration_s":0.069768,"completed_at":"2026-09-01T10:35:08.194Z","load_balancing_strategy":"primary","db_duration_s":0.015068,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1514gitlab # [ 237.047254] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:08.205Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"dd35bab49079fdf67d184187","created_at":"2026-09-01T10:35:02.445Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d55d82fa3ae7fbb95f8ae08f1ac821c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.449Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-dd35bab49079fdf67d184187: start","job_status":"start","queue_duration_s":5.755724,"scheduling_latency_s":5.755768}1515gitlab # [ 238.344257] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:09.501Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"dd35bab49079fdf67d184187","created_at":"2026-09-01T10:35:02.445Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d55d82fa3ae7fbb95f8ae08f1ac821c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.449Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-dd35bab49079fdf67d184187: done: 1.296516 sec","job_status":"done","queue_duration_s":5.755724,"scheduling_latency_s":5.755768,"redis_calls":9,"redis_duration_s":0.022382,"redis_read_bytes":7,"redis_write_bytes":987,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005588,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.016794,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":129,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":129,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.438,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.438,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.353128,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000974425,"exclusive_lock_hold_duration_s":1.242586139,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":1.296516,"completed_at":"2026-09-01T10:35:09.501Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.769895,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1516gitlab # [ 238.419701] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:09.577Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"c371d02db737c4f5d8ec67bf","created_at":"2026-09-01T10:35:02.463Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c0f988e51171befda054860cefbeffb7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.467Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-c371d02db737c4f5d8ec67bf: start","job_status":"start","queue_duration_s":7.109327,"scheduling_latency_s":7.109371}1517gitlab # [ 238.497259] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:09.654Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"c371d02db737c4f5d8ec67bf","created_at":"2026-09-01T10:35:02.463Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c0f988e51171befda054860cefbeffb7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:02.467Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-c371d02db737c4f5d8ec67bf: done: 0.077467 sec","job_status":"done","queue_duration_s":7.109327,"scheduling_latency_s":7.109371,"redis_calls":7,"redis_duration_s":0.01569,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.010975,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004715,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017792,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.077467,"completed_at":"2026-09-01T10:35:09.654Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1518gitlab # [ 238.575458] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:09.733Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"9d42be29823b3c17092ab61a","created_at":"2026-09-01T10:35:02.484Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"480f81520fb4a18f6f61e92280f79b19","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-01T10:35:02.489Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-9d42be29823b3c17092ab61a: start","job_status":"start","queue_duration_s":7.243957,"scheduling_latency_s":7.244}1519gitlab # [ 238.645144] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:09.801Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"9d42be29823b3c17092ab61a","created_at":"2026-09-01T10:35:02.484Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"480f81520fb4a18f6f61e92280f79b19","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-01T10:35:02.489Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-9d42be29823b3c17092ab61a: done: 0.068665 sec","job_status":"done","queue_duration_s":7.243957,"scheduling_latency_s":7.244,"redis_calls":7,"redis_duration_s":0.012335,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.009346,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002989,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022191,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.068665,"completed_at":"2026-09-01T10:35:09.801Z","load_balancing_strategy":"primary","db_duration_s":0.003458,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1520gitlab # [ 238.708762] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:09.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"041966a158e8d798d9384b62","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-041966a158e8d798d9384b62: start","job_status":"start","queue_duration_s":4.84819,"scheduling_latency_s":4.848246}1521gitlab # [ 238.728604] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:09.871Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"041966a158e8d798d9384b62","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-041966a158e8d798d9384b62: done: 0.064518 sec","job_status":"done","queue_duration_s":4.84819,"scheduling_latency_s":4.848246,"redis_calls":13,"redis_duration_s":0.019367,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.010884,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008483,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026019,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.064518,"completed_at":"2026-09-01T10:35:09.871Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1522gitlab # [ 238.787539] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:09.876Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6e23558cffc9b7d5c0f9fc9","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6e23558cffc9b7d5c0f9fc9: start","job_status":"start","queue_duration_s":4.917473,"scheduling_latency_s":4.917518}1523gitlab # [ 238.818964] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:09.975Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6e23558cffc9b7d5c0f9fc9","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6e23558cffc9b7d5c0f9fc9: done: 0.099206 sec","job_status":"done","queue_duration_s":4.917473,"scheduling_latency_s":4.917518,"redis_calls":13,"redis_duration_s":0.063078,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.007258,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.05582,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.099206,"completed_at":"2026-09-01T10:35:09.975Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1524gitlab # [ 238.907208] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:10.062Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a6427b9d63186b9d414da72","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a6427b9d63186b9d414da72: start","job_status":"start","queue_duration_s":5.103502,"scheduling_latency_s":5.103543}1525gitlab # [ 239.017400] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:10.175Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a6427b9d63186b9d414da72","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a6427b9d63186b9d414da72: done: 0.11244 sec","job_status":"done","queue_duration_s":5.103502,"scheduling_latency_s":5.103543,"redis_calls":13,"redis_duration_s":0.031877,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.007004,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.024873,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019636,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.11244,"completed_at":"2026-09-01T10:35:10.175Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1526gitlab # [ 239.080266] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:10.236Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6ddfdfbb180dedd5a4f8240","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6ddfdfbb180dedd5a4f8240: start","job_status":"start","queue_duration_s":5.277597,"scheduling_latency_s":5.277644}1527gitlab # [ 239.145865] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:10.302Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6ddfdfbb180dedd5a4f8240","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6ddfdfbb180dedd5a4f8240: done: 0.065372 sec","job_status":"done","queue_duration_s":5.277597,"scheduling_latency_s":5.277644,"redis_calls":13,"redis_duration_s":0.023233999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00828,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.014954,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015626,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.065372,"completed_at":"2026-09-01T10:35:10.302Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1528gitlab # [ 239.223655] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:10.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d18156e65dee11fa3e84cad","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d18156e65dee11fa3e84cad: start","job_status":"start","queue_duration_s":5.419444,"scheduling_latency_s":5.419486}1529gitlab # [ 239.312518] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:10.469Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d18156e65dee11fa3e84cad","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d18156e65dee11fa3e84cad: done: 0.090509 sec","job_status":"done","queue_duration_s":5.419444,"scheduling_latency_s":5.419486,"redis_calls":13,"redis_duration_s":0.026164,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.021247,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.090509,"completed_at":"2026-09-01T10:35:10.469Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1530gitlab # [ 239.403994] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:10.556Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9015243f747944a726764be7","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9015243f747944a726764be7: start","job_status":"start","queue_duration_s":5.597221,"scheduling_latency_s":5.597264}1531gitlab # [ 239.480694] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:10.637Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9015243f747944a726764be7","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9015243f747944a726764be7: done: 0.080885 sec","job_status":"done","queue_duration_s":5.597221,"scheduling_latency_s":5.597264,"redis_calls":13,"redis_duration_s":0.023414,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00906,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.014354,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018421,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.080885,"completed_at":"2026-09-01T10:35:10.637Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1532gitlab # [ 239.596730] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:10.643Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7eb187665e67d64ceb7c46a","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7eb187665e67d64ceb7c46a: start","job_status":"start","queue_duration_s":5.684135,"scheduling_latency_s":5.684188}1533gitlab # [ 239.672163] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:10.823Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7eb187665e67d64ceb7c46a","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7eb187665e67d64ceb7c46a: done: 0.18076 sec","job_status":"done","queue_duration_s":5.684135,"scheduling_latency_s":5.684188,"redis_calls":13,"redis_duration_s":0.138069,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.017413,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.120656,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017634,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.18076,"completed_at":"2026-09-01T10:35:10.823Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1534gitlab # [ 239.680811] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:10.832Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e226ff97891cf97d28bd83f9","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e226ff97891cf97d28bd83f9: start","job_status":"start","queue_duration_s":5.873464,"scheduling_latency_s":5.873512}1535gitlab # [ 239.786566] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:10.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e226ff97891cf97d28bd83f9","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e226ff97891cf97d28bd83f9: done: 0.100861 sec","job_status":"done","queue_duration_s":5.873464,"scheduling_latency_s":5.873512,"redis_calls":13,"redis_duration_s":0.012638,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.008417,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004221,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018692,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.100861,"completed_at":"2026-09-01T10:35:10.933Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1536gitlab # [ 239.788798] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:10.937Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"427a908224fa59ee80a1c18a","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-427a908224fa59ee80a1c18a: start","job_status":"start","queue_duration_s":5.9778,"scheduling_latency_s":5.977844}1537gitlab # [ 240.150523] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:11.308Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"427a908224fa59ee80a1c18a","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-427a908224fa59ee80a1c18a: done: 0.371101 sec","job_status":"done","queue_duration_s":5.9778,"scheduling_latency_s":5.977844,"redis_calls":13,"redis_duration_s":0.285609,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.007089,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.27852,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.371101,"completed_at":"2026-09-01T10:35:11.307Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1538gitlab # [ 240.229861] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:11.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"166bdbae4ad797dce730cf02","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-166bdbae4ad797dce730cf02: start","job_status":"start","queue_duration_s":6.427831,"scheduling_latency_s":6.42788}1539gitlab # [ 240.316922] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:11.473Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"166bdbae4ad797dce730cf02","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-166bdbae4ad797dce730cf02: done: 0.086553 sec","job_status":"done","queue_duration_s":6.427831,"scheduling_latency_s":6.42788,"redis_calls":13,"redis_duration_s":0.025058,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00894,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.016118,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.086553,"completed_at":"2026-09-01T10:35:11.473Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1540gitlab # [ 240.390479] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:11.545Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5585a8c4f92556da79e9eaea","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5585a8c4f92556da79e9eaea: start","job_status":"start","queue_duration_s":6.586165,"scheduling_latency_s":6.58621}1541gitlab # [ 240.461992] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:11.618Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5585a8c4f92556da79e9eaea","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5585a8c4f92556da79e9eaea: done: 0.073652 sec","job_status":"done","queue_duration_s":6.586165,"scheduling_latency_s":6.58621,"redis_calls":13,"redis_duration_s":0.028759,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002077,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.026682,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.073652,"completed_at":"2026-09-01T10:35:11.618Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1542gitlab # [ 240.484168] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:11.641Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44d4164dc298693a70b55212","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44d4164dc298693a70b55212: start","job_status":"start","queue_duration_s":6.681808,"scheduling_latency_s":6.681855}1543gitlab # [ 240.566842] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:11.724Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44d4164dc298693a70b55212","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44d4164dc298693a70b55212: done: 0.082981 sec","job_status":"done","queue_duration_s":6.681808,"scheduling_latency_s":6.681855,"redis_calls":13,"redis_duration_s":0.029771,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002085,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.027686,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.082981,"completed_at":"2026-09-01T10:35:11.723Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1544gitlab # [ 240.573493] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:11.730Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75638e74616b288adf2774d3","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75638e74616b288adf2774d3: start","job_status":"start","queue_duration_s":6.771128,"scheduling_latency_s":6.771174}1545gitlab # [ 240.669776] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:11.796Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75638e74616b288adf2774d3","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75638e74616b288adf2774d3: done: 0.066448 sec","job_status":"done","queue_duration_s":6.771128,"scheduling_latency_s":6.771174,"redis_calls":13,"redis_duration_s":0.0050349999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003519,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021438,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.066448,"completed_at":"2026-09-01T10:35:11.796Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1546gitlab # [ 240.747060] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:11.801Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32d41a534b22ce45737d9bb6","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32d41a534b22ce45737d9bb6: start","job_status":"start","queue_duration_s":6.841956,"scheduling_latency_s":6.842003}1547gitlab # [ 240.753106] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:11.865Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32d41a534b22ce45737d9bb6","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32d41a534b22ce45737d9bb6: done: 0.063921 sec","job_status":"done","queue_duration_s":6.841956,"scheduling_latency_s":6.842003,"redis_calls":13,"redis_duration_s":0.015059,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001349,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.01371,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.063921,"completed_at":"2026-09-01T10:35:11.864Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1548gitlab # [ 240.824712] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:11.869Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"efdaf0a535563f6ca9776ede","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-efdaf0a535563f6ca9776ede: start","job_status":"start","queue_duration_s":6.91062,"scheduling_latency_s":6.910661}1549gitlab # [ 240.825124] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:11.924Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"efdaf0a535563f6ca9776ede","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-efdaf0a535563f6ca9776ede: done: 0.054914 sec","job_status":"done","queue_duration_s":6.91062,"scheduling_latency_s":6.910661,"redis_calls":13,"redis_duration_s":0.007339999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003691,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003649,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017556,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.054914,"completed_at":"2026-09-01T10:35:11.924Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1550gitlab # [ 240.896450] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:11.929Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95ccac9de3e86fc683eb30d1","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95ccac9de3e86fc683eb30d1: start","job_status":"start","queue_duration_s":6.970207,"scheduling_latency_s":6.970251}1551gitlab # [ 241.043698] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:12.201Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95ccac9de3e86fc683eb30d1","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95ccac9de3e86fc683eb30d1: done: 0.27178 sec","job_status":"done","queue_duration_s":6.970207,"scheduling_latency_s":6.970251,"redis_calls":13,"redis_duration_s":0.174597,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.131691,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.042906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007581,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.27178,"completed_at":"2026-09-01T10:35:12.201Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1552gitlab # [ 241.187666] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:12.345Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54965f31bb2d5126c07fd7ce","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54965f31bb2d5126c07fd7ce: start","job_status":"start","queue_duration_s":7.386194,"scheduling_latency_s":7.386239}1553gitlab # [ 241.232474] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:12.389Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54965f31bb2d5126c07fd7ce","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54965f31bb2d5126c07fd7ce: done: 0.043767 sec","job_status":"done","queue_duration_s":7.386194,"scheduling_latency_s":7.386239,"redis_calls":13,"redis_duration_s":0.00482,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001683,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003137,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.043767,"completed_at":"2026-09-01T10:35:12.388Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1554gitlab # [ 241.235490] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:12.392Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de30265fa0828c66c8428030","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de30265fa0828c66c8428030: start","job_status":"start","queue_duration_s":7.433424,"scheduling_latency_s":7.433465}1555gitlab # [ 241.428279] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:12.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de30265fa0828c66c8428030","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de30265fa0828c66c8428030: done: 0.048915 sec","job_status":"done","queue_duration_s":7.433424,"scheduling_latency_s":7.433465,"redis_calls":13,"redis_duration_s":0.018556000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.010512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008044,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.048915,"completed_at":"2026-09-01T10:35:12.441Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1556gitlab # [ 241.555414] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:12.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9215cf27d42be4d547fa48a8","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9215cf27d42be4d547fa48a8: start","job_status":"start","queue_duration_s":7.493543,"scheduling_latency_s":7.493589}1557gitlab # [ 241.555724] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:12.488Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9215cf27d42be4d547fa48a8","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9215cf27d42be4d547fa48a8: done: 0.036117 sec","job_status":"done","queue_duration_s":7.493543,"scheduling_latency_s":7.493589,"redis_calls":13,"redis_duration_s":0.005881000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002109,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003772,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022354,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.036117,"completed_at":"2026-09-01T10:35:12.488Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1558gitlab # [ 241.697543] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:12.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4ad7ad2a1752fc801def11b","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4ad7ad2a1752fc801def11b: start","job_status":"start","queue_duration_s":7.532466,"scheduling_latency_s":7.532507}1559gitlab # [ 241.698859] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:12.532Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4ad7ad2a1752fc801def11b","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4ad7ad2a1752fc801def11b: done: 0.040517 sec","job_status":"done","queue_duration_s":7.532466,"scheduling_latency_s":7.532507,"redis_calls":13,"redis_duration_s":0.01022,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004725,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005495,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02126,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.040517,"completed_at":"2026-09-01T10:35:12.532Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1560gitlab # [ 241.830005] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:12.542Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d39c7aa39a9b1830f713fb2","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d39c7aa39a9b1830f713fb2: start","job_status":"start","queue_duration_s":7.582886,"scheduling_latency_s":7.582933}1561gitlab # [ 241.831277] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:12.590Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d39c7aa39a9b1830f713fb2","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d39c7aa39a9b1830f713fb2: done: 0.048309 sec","job_status":"done","queue_duration_s":7.582886,"scheduling_latency_s":7.582933,"redis_calls":13,"redis_duration_s":0.004281,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001136,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003145,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019193,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.048309,"completed_at":"2026-09-01T10:35:12.590Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1562gitlab # [ 242.024947] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:12.629Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ca1c7575424693cdf113646","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ca1c7575424693cdf113646: start","job_status":"start","queue_duration_s":7.670273,"scheduling_latency_s":7.670322}1563gitlab # [ 242.025399] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:12.718Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ca1c7575424693cdf113646","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ca1c7575424693cdf113646: done: 0.088884 sec","job_status":"done","queue_duration_s":7.670273,"scheduling_latency_s":7.670322,"redis_calls":13,"redis_duration_s":0.008194,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002784,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00541,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0315,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.088884,"completed_at":"2026-09-01T10:35:12.718Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1564gitlab # [ 242.168537] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:12.724Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7592d88c789c763fdae3e266","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7592d88c789c763fdae3e266: start","job_status":"start","queue_duration_s":7.765403,"scheduling_latency_s":7.765454}1565gitlab # [ 242.184710] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:12.794Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7592d88c789c763fdae3e266","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7592d88c789c763fdae3e266: done: 0.070461 sec","job_status":"done","queue_duration_s":7.765403,"scheduling_latency_s":7.765454,"redis_calls":13,"redis_duration_s":0.033902,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.012592,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.02131,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02304,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.070461,"completed_at":"2026-09-01T10:35:12.794Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1566gitlab # [ 242.340385] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:12.799Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea1529d5c1cda830d44bbdbb","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea1529d5c1cda830d44bbdbb: start","job_status":"start","queue_duration_s":7.839821,"scheduling_latency_s":7.839868}1567gitlab # [ 242.341544] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:12.832Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea1529d5c1cda830d44bbdbb","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea1529d5c1cda830d44bbdbb: done: 0.033334 sec","job_status":"done","queue_duration_s":7.839821,"scheduling_latency_s":7.839868,"redis_calls":13,"redis_duration_s":0.00664,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001884,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004756,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033334,"completed_at":"2026-09-01T10:35:12.832Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1568gitlab # [ 242.446923] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:12.835Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fc710911ab5bb1302114d1d","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fc710911ab5bb1302114d1d: start","job_status":"start","queue_duration_s":7.875981,"scheduling_latency_s":7.876019}1569gitlab # [ 242.447358] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:12.865Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fc710911ab5bb1302114d1d","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fc710911ab5bb1302114d1d: done: 0.030428 sec","job_status":"done","queue_duration_s":7.875981,"scheduling_latency_s":7.876019,"redis_calls":13,"redis_duration_s":0.005716,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001678,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030428,"completed_at":"2026-09-01T10:35:12.865Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1570gitlab # [ 242.582809] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:12.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"769653eaa3265b9de951e79b","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-769653eaa3265b9de951e79b: start","job_status":"start","queue_duration_s":7.908721,"scheduling_latency_s":7.908758}1571gitlab # [ 242.658703] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:12.916Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"769653eaa3265b9de951e79b","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-769653eaa3265b9de951e79b: done: 0.048876 sec","job_status":"done","queue_duration_s":7.908721,"scheduling_latency_s":7.908758,"redis_calls":13,"redis_duration_s":0.013147,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.008382,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004765,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021513,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.048876,"completed_at":"2026-09-01T10:35:12.916Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1572gitlab # [ 242.660975] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:12.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77c9d68de3a7e357559f92cd","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77c9d68de3a7e357559f92cd: start","job_status":"start","queue_duration_s":7.960518,"scheduling_latency_s":7.960563}1573gitlab # [ 242.672283] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.034Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77c9d68de3a7e357559f92cd","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77c9d68de3a7e357559f92cd: done: 0.115072 sec","job_status":"done","queue_duration_s":7.960518,"scheduling_latency_s":7.960563,"redis_calls":13,"redis_duration_s":0.038801999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004003,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.034799,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024232,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.115072,"completed_at":"2026-09-01T10:35:13.034Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1574gitlab # [ 242.731296] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.050Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6041c0bf15258b40912dfb02","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6041c0bf15258b40912dfb02: start","job_status":"start","queue_duration_s":8.091298,"scheduling_latency_s":8.091364}1575gitlab # [ 242.836198] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.103Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6041c0bf15258b40912dfb02","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6041c0bf15258b40912dfb02: done: 0.060656 sec","job_status":"done","queue_duration_s":8.091298,"scheduling_latency_s":8.091364,"redis_calls":13,"redis_duration_s":0.008123,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00325,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004873,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02035,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.060656,"completed_at":"2026-09-01T10:35:13.103Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1576gitlab # [ 242.842408] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.107Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a96ae4568743d80dfa15eb8","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a96ae4568743d80dfa15eb8: start","job_status":"start","queue_duration_s":8.147874,"scheduling_latency_s":8.147921}1577gitlab # [ 242.923637] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.146Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a96ae4568743d80dfa15eb8","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a96ae4568743d80dfa15eb8: done: 0.03959 sec","job_status":"done","queue_duration_s":8.147874,"scheduling_latency_s":8.147921,"redis_calls":13,"redis_duration_s":0.008543,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006049,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03959,"completed_at":"2026-09-01T10:35:13.146Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1578gitlab # [ 242.951389] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.151Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e391453b3133f137a0fad659","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e391453b3133f137a0fad659: start","job_status":"start","queue_duration_s":8.192413,"scheduling_latency_s":8.192463}1579gitlab # [ 243.094316] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.187Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e391453b3133f137a0fad659","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e391453b3133f137a0fad659: done: 0.035872 sec","job_status":"done","queue_duration_s":8.192413,"scheduling_latency_s":8.192463,"redis_calls":13,"redis_duration_s":0.006821000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00218,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004641,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035872,"completed_at":"2026-09-01T10:35:13.187Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1580gitlab # [ 243.111322] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1e68f4408ff07aee416ad69","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1e68f4408ff07aee416ad69: start","job_status":"start","queue_duration_s":8.246586,"scheduling_latency_s":8.246631}1581gitlab # [ 243.365166] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.303Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1e68f4408ff07aee416ad69","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1e68f4408ff07aee416ad69: done: 0.097907 sec","job_status":"done","queue_duration_s":8.246586,"scheduling_latency_s":8.246631,"redis_calls":13,"redis_duration_s":0.003827,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001235,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002592,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016047,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.097907,"completed_at":"2026-09-01T10:35:13.303Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1582gitlab # [ 243.441194] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.305Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb0bc134c04c6b8acfdfee5b","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb0bc134c04c6b8acfdfee5b: start","job_status":"start","queue_duration_s":8.346711,"scheduling_latency_s":8.346747}1583gitlab # [ 243.441536] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.368Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb0bc134c04c6b8acfdfee5b","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb0bc134c04c6b8acfdfee5b: done: 0.06221 sec","job_status":"done","queue_duration_s":8.346711,"scheduling_latency_s":8.346747,"redis_calls":13,"redis_duration_s":0.018430000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004364,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.014066,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021899,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.06221,"completed_at":"2026-09-01T10:35:13.368Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1584gitlab # [ 243.508293] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9f831019b881e4c637915e3","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9f831019b881e4c637915e3: start","job_status":"start","queue_duration_s":8.419171,"scheduling_latency_s":8.419225}1585gitlab # [ 243.511566] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9f831019b881e4c637915e3","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9f831019b881e4c637915e3: done: 0.113016 sec","job_status":"done","queue_duration_s":8.419171,"scheduling_latency_s":8.419225,"redis_calls":13,"redis_duration_s":0.0494,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.010751,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.038649,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02655,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.113016,"completed_at":"2026-09-01T10:35:13.491Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1586gitlab # [ 243.543379] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.496Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d92b80e827d0acd553bcccc9","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d92b80e827d0acd553bcccc9: start","job_status":"start","queue_duration_s":8.537071,"scheduling_latency_s":8.537198}1587gitlab # [ 243.626268] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.588Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d92b80e827d0acd553bcccc9","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d92b80e827d0acd553bcccc9: done: 0.092043 sec","job_status":"done","queue_duration_s":8.537071,"scheduling_latency_s":8.537198,"redis_calls":13,"redis_duration_s":0.023918000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.008754,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.015164,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021759,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.092043,"completed_at":"2026-09-01T10:35:13.588Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1588gitlab # [ 243.631570] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.590Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bc9e2da9396ec526b5ddde3","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bc9e2da9396ec526b5ddde3: start","job_status":"start","queue_duration_s":8.631601,"scheduling_latency_s":8.631634}1589gitlab # [ 243.732989] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.629Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bc9e2da9396ec526b5ddde3","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bc9e2da9396ec526b5ddde3: done: 0.038695 sec","job_status":"done","queue_duration_s":8.631601,"scheduling_latency_s":8.631634,"redis_calls":13,"redis_duration_s":0.012362999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005379,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006984,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019139,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038695,"completed_at":"2026-09-01T10:35:13.629Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1590gitlab # [ 243.899614] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.632Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b85a8cf541733211eae7e112","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b85a8cf541733211eae7e112: start","job_status":"start","queue_duration_s":8.673345,"scheduling_latency_s":8.673386}1591gitlab # [ 243.900056] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.691Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b85a8cf541733211eae7e112","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b85a8cf541733211eae7e112: done: 0.058752 sec","job_status":"done","queue_duration_s":8.673345,"scheduling_latency_s":8.673386,"redis_calls":13,"redis_duration_s":0.020256,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00691,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.013346,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.058752,"completed_at":"2026-09-01T10:35:13.691Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1592gitlab # [ 244.043232] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.699Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b336badad3fce36bd5bb2b5","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b336badad3fce36bd5bb2b5: start","job_status":"start","queue_duration_s":8.739878,"scheduling_latency_s":8.739926}1593gitlab # [ 244.043574] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b336badad3fce36bd5bb2b5","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b336badad3fce36bd5bb2b5: done: 0.044688 sec","job_status":"done","queue_duration_s":8.739878,"scheduling_latency_s":8.739926,"redis_calls":13,"redis_duration_s":0.01141,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004455,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006955,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020738,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.044688,"completed_at":"2026-09-01T10:35:13.743Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1594gitlab # [ 244.117908] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.749Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ffca3798d3a71496147cb7a9","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ffca3798d3a71496147cb7a9: start","job_status":"start","queue_duration_s":8.790377,"scheduling_latency_s":8.790432}1595gitlab # [ 244.119732] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.781Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ffca3798d3a71496147cb7a9","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ffca3798d3a71496147cb7a9: done: 0.031901 sec","job_status":"done","queue_duration_s":8.790377,"scheduling_latency_s":8.790432,"redis_calls":13,"redis_duration_s":0.00666,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002262,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004398,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031901,"completed_at":"2026-09-01T10:35:13.781Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1596gitlab # [ 244.192710] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.784Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d491fc187cf9d325e003dd02","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d491fc187cf9d325e003dd02: start","job_status":"start","queue_duration_s":8.825254,"scheduling_latency_s":8.8253}1597gitlab # [ 244.193143] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.814Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d491fc187cf9d325e003dd02","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d491fc187cf9d325e003dd02: done: 0.029845 sec","job_status":"done","queue_duration_s":8.825254,"scheduling_latency_s":8.8253,"redis_calls":13,"redis_duration_s":0.005371,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001752,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003619,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017422,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029845,"completed_at":"2026-09-01T10:35:13.814Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1598gitlab # [ 244.264554] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cea02c17b70339d7287d094d","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cea02c17b70339d7287d094d: start","job_status":"start","queue_duration_s":8.857377,"scheduling_latency_s":8.857419}1599gitlab # [ 244.267470] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.840Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cea02c17b70339d7287d094d","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cea02c17b70339d7287d094d: done: 0.023507 sec","job_status":"done","queue_duration_s":8.857377,"scheduling_latency_s":8.857419,"redis_calls":13,"redis_duration_s":0.003528,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001139,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002389,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023507,"completed_at":"2026-09-01T10:35:13.839Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1600gitlab # [ 244.332370] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.842Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5218a795faacad8bc37b969","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5218a795faacad8bc37b969: start","job_status":"start","queue_duration_s":8.883223,"scheduling_latency_s":8.883263}1601gitlab # [ 244.340562] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5218a795faacad8bc37b969","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5218a795faacad8bc37b969: done: 0.024793 sec","job_status":"done","queue_duration_s":8.883223,"scheduling_latency_s":8.883263,"redis_calls":13,"redis_duration_s":0.003593,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001138,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002455,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024793,"completed_at":"2026-09-01T10:35:13.867Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1602gitlab # [ 244.445006] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.869Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"add62cdebdb8fa84f4680092","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-add62cdebdb8fa84f4680092: start","job_status":"start","queue_duration_s":8.910165,"scheduling_latency_s":8.910201}1603gitlab # [ 244.662982] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.895Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"add62cdebdb8fa84f4680092","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-add62cdebdb8fa84f4680092: done: 0.025995 sec","job_status":"done","queue_duration_s":8.910165,"scheduling_latency_s":8.910201,"redis_calls":13,"redis_duration_s":0.0040160000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001594,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002422,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016255,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025995,"completed_at":"2026-09-01T10:35:13.895Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1604gitlab # [ 244.663321] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.897Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35cbaa8c1e0fe2cdecd88344","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35cbaa8c1e0fe2cdecd88344: start","job_status":"start","queue_duration_s":8.93841,"scheduling_latency_s":8.93845}1605gitlab # [ 244.974629] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35cbaa8c1e0fe2cdecd88344","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35cbaa8c1e0fe2cdecd88344: done: 0.026064 sec","job_status":"done","queue_duration_s":8.93841,"scheduling_latency_s":8.93845,"redis_calls":13,"redis_duration_s":0.004402,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001245,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003157,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01675,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026064,"completed_at":"2026-09-01T10:35:13.923Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1606gitlab # [ 245.242481] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.926Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0cdf0f27846bf8afeb7cb2d","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0cdf0f27846bf8afeb7cb2d: start","job_status":"start","queue_duration_s":8.966988,"scheduling_latency_s":8.967025}1607gitlab # [ 245.245487] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.954Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0cdf0f27846bf8afeb7cb2d","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0cdf0f27846bf8afeb7cb2d: done: 0.02874 sec","job_status":"done","queue_duration_s":8.966988,"scheduling_latency_s":8.967025,"redis_calls":13,"redis_duration_s":0.004794,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001651,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003143,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02874,"completed_at":"2026-09-01T10:35:13.954Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1608gitlab # [ 245.545391] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:13.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9ba5c02989cd6aed82e15e0","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9ba5c02989cd6aed82e15e0: start","job_status":"start","queue_duration_s":8.998205,"scheduling_latency_s":8.99825}1609gitlab # [ 245.545800] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:14.020Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9ba5c02989cd6aed82e15e0","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9ba5c02989cd6aed82e15e0: done: 0.063217 sec","job_status":"done","queue_duration_s":8.998205,"scheduling_latency_s":8.99825,"redis_calls":13,"redis_duration_s":0.0069949999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001945,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00505,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.063217,"completed_at":"2026-09-01T10:35:14.020Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1610gitlab # [ 245.775538] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:14.023Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f270de8b4263b24c2fb27dcb","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f270de8b4263b24c2fb27dcb: start","job_status":"start","queue_duration_s":9.064538,"scheduling_latency_s":9.064581}1611gitlab # [ 245.776863] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:14.225Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f270de8b4263b24c2fb27dcb","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f270de8b4263b24c2fb27dcb: done: 0.201535 sec","job_status":"done","queue_duration_s":9.064538,"scheduling_latency_s":9.064581,"redis_calls":13,"redis_duration_s":0.048089,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.026377,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.021712,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025395,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.201535,"completed_at":"2026-09-01T10:35:14.225Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1612gitlab # [ 245.993585] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:14.231Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e46402ebce0096d5df57cbab","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e46402ebce0096d5df57cbab: start","job_status":"start","queue_duration_s":9.27258,"scheduling_latency_s":9.272623}1613gitlab # [ 245.997332] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:14.308Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e46402ebce0096d5df57cbab","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e46402ebce0096d5df57cbab: done: 0.07632 sec","job_status":"done","queue_duration_s":9.27258,"scheduling_latency_s":9.272623,"redis_calls":13,"redis_duration_s":0.032492,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.012917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.019575,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.07632,"completed_at":"2026-09-01T10:35:14.307Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1614gitlab # [ 246.544351] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:14.318Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab81e8537a41fe307e8f1d1c","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab81e8537a41fe307e8f1d1c: start","job_status":"start","queue_duration_s":9.359674,"scheduling_latency_s":9.359721}1615gitlab # [ 246.923048] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:14.707Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab81e8537a41fe307e8f1d1c","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab81e8537a41fe307e8f1d1c: done: 0.389014 sec","job_status":"done","queue_duration_s":9.359674,"scheduling_latency_s":9.359721,"redis_calls":13,"redis_duration_s":0.320682,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.010488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.310194,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025242,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.389014,"completed_at":"2026-09-01T10:35:14.707Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1616gitlab # [ 246.955575] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:14.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98fd67fc8eca1b080e607b0f","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98fd67fc8eca1b080e607b0f: start","job_status":"start","queue_duration_s":9.757507,"scheduling_latency_s":9.757553}1617gitlab # [ 247.067687] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:14.742Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98fd67fc8eca1b080e607b0f","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98fd67fc8eca1b080e607b0f: done: 0.025615 sec","job_status":"done","queue_duration_s":9.757507,"scheduling_latency_s":9.757553,"redis_calls":13,"redis_duration_s":0.003855,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001188,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002667,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025615,"completed_at":"2026-09-01T10:35:14.742Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1618gitlab # [ 247.070146] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:14.744Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"495221339a4e3e9eda0b35bd","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-495221339a4e3e9eda0b35bd: start","job_status":"start","queue_duration_s":9.785462,"scheduling_latency_s":9.785503}1619gitlab # [ 247.108469] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:14.769Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"495221339a4e3e9eda0b35bd","created_at":"2026-09-01T10:35:04.564Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94fc1e33ad154499","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b30d92cf4e9fc6cc4b9db5faa77ee35e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:04.959Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-495221339a4e3e9eda0b35bd: done: 0.024562 sec","job_status":"done","queue_duration_s":9.785462,"scheduling_latency_s":9.785503,"redis_calls":13,"redis_duration_s":0.003752,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001114,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002638,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024562,"completed_at":"2026-09-01T10:35:14.769Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1620gitlab # [ 247.199398] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:14.771Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"72967754bbcde63b0227ee5a","created_at":"2026-09-01T10:35:06.223Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-2681aa77767046bb","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3df57e7460e46f76d16a1a6e0bb017ea","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.353Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-72967754bbcde63b0227ee5a: start","job_status":"start","queue_duration_s":8.417405,"scheduling_latency_s":8.417443}1621gitlab # [ 247.200753] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:14.842Z","message":"No existing merge request to be cleaned up."}1622gitlab # [ 247.325434] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:14.880Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"72967754bbcde63b0227ee5a","created_at":"2026-09-01T10:35:06.223Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-2681aa77767046bb","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3df57e7460e46f76d16a1a6e0bb017ea","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.353Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-72967754bbcde63b0227ee5a: done: 0.10947 sec","job_status":"done","queue_duration_s":8.417405,"scheduling_latency_s":8.417443,"redis_calls":10,"redis_duration_s":0.047418,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.025921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.021497,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.10947,"completed_at":"2026-09-01T10:35:14.880Z","load_balancing_strategy":"primary","db_duration_s":0.018232,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1623gitlab # [ 247.325798] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:14.896Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fe8dff028700f6bab353545c","created_at":"2026-09-01T10:35:06.223Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-2681aa77767046bb","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3df57e7460e46f76d16a1a6e0bb017ea","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.353Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fe8dff028700f6bab353545c: start","job_status":"start","queue_duration_s":8.542065,"scheduling_latency_s":8.54211}1624gitlab # [ 247.442103] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:14.944Z","message":"No existing merge request to be cleaned up."}1625gitlab # [ 247.452996] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:14.995Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fe8dff028700f6bab353545c","created_at":"2026-09-01T10:35:06.223Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-2681aa77767046bb","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3df57e7460e46f76d16a1a6e0bb017ea","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.353Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fe8dff028700f6bab353545c: done: 0.099197 sec","job_status":"done","queue_duration_s":8.542065,"scheduling_latency_s":8.54211,"redis_calls":10,"redis_duration_s":0.023195,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.014198,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.008997,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028559,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.099197,"completed_at":"2026-09-01T10:35:14.995Z","load_balancing_strategy":"primary","db_duration_s":0.015978,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1626gitlab # [ 247.667531] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:15.018Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"71ac3c2bbc5a247ca6d89b80","created_at":"2026-09-01T10:35:06.223Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-2681aa77767046bb","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3df57e7460e46f76d16a1a6e0bb017ea","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.353Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-71ac3c2bbc5a247ca6d89b80: start","job_status":"start","queue_duration_s":8.66326,"scheduling_latency_s":8.663313}1627gitlab # [ 247.669055] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:15.096Z","message":"No existing merge request to be cleaned up."}1628gitlab # [ 247.670073] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:15.150Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"71ac3c2bbc5a247ca6d89b80","created_at":"2026-09-01T10:35:06.223Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-2681aa77767046bb","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3df57e7460e46f76d16a1a6e0bb017ea","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.353Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-71ac3c2bbc5a247ca6d89b80: done: 0.13388 sec","job_status":"done","queue_duration_s":8.66326,"scheduling_latency_s":8.663313,"redis_calls":10,"redis_duration_s":0.024855000000000002,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.011677,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.013178,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.015,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.015,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.13388,"completed_at":"2026-09-01T10:35:15.150Z","load_balancing_strategy":"primary","db_duration_s":0.030257,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1629gitlab # [ 248.099631] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:15.153Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ac39aad0db902b5387b43634","created_at":"2026-09-01T10:35:06.223Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-2681aa77767046bb","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3df57e7460e46f76d16a1a6e0bb017ea","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.353Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ac39aad0db902b5387b43634: start","job_status":"start","queue_duration_s":8.799975,"scheduling_latency_s":8.800015}1630gitlab # [ 248.103639] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:15.175Z","message":"No existing merge request to be cleaned up."}1631gitlab # [ 248.106604] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:35:15.195Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ac39aad0db902b5387b43634","created_at":"2026-09-01T10:35:06.223Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-2681aa77767046bb","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3df57e7460e46f76d16a1a6e0bb017ea","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.353Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ac39aad0db902b5387b43634: done: 0.041566 sec","job_status":"done","queue_duration_s":8.799975,"scheduling_latency_s":8.800015,"redis_calls":10,"redis_duration_s":0.0059229999999999994,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003071,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002852,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041566,"completed_at":"2026-09-01T10:35:15.195Z","load_balancing_strategy":"primary","db_duration_s":0.003611,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1632gitlab # [ 261.784526] hrtimer: interrupt took 26394695 ns1633gitlab # [ 290.491566] postgres[3018]: [3018] ERROR: relation "table_without_models" does not exist1634gitlab # [ 290.492647] postgres[3018]: [3018] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1635gitlab # [ 293.057179] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:04.214Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"6f4df7f4a5678cec07cdda6c","created_at":"2026-09-01T10:36:04.181Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4df108f43010d85bfcfe84d769916d64","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.212Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-6f4df7f4a5678cec07cdda6c: start","job_status":"start","queue_duration_s":0.001652,"scheduling_latency_s":0.001685}1636gitlab # [ 293.117371] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:04.275Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"6f4df7f4a5678cec07cdda6c","created_at":"2026-09-01T10:36:04.181Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4df108f43010d85bfcfe84d769916d64","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.212Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-6f4df7f4a5678cec07cdda6c: done: 0.060612 sec","job_status":"done","queue_duration_s":0.001652,"scheduling_latency_s":0.001685,"redis_calls":12,"redis_duration_s":0.011663,"redis_read_bytes":267,"redis_write_bytes":1280,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.002972,"redis_feature_flag_read_bytes":260,"redis_feature_flag_write_bytes":153,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006054,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002637,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019528,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001720051,"exclusive_lock_hold_duration_s":0.028715661,"duration_s":0.060612,"completed_at":"2026-09-01T10:36:04.275Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.034757,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1637gitlab # [ 293.180626] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:04.338Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"7ee2fb185d4d29e637ab7aa9","created_at":"2026-09-01T10:36:04.229Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f441dd7f7a124867628afc730b5d243","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-01T10:36:04.237Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-7ee2fb185d4d29e637ab7aa9: start","job_status":"start","queue_duration_s":0.100707,"scheduling_latency_s":0.10074}1638gitlab # [ 293.225225] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:04.383Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1639gitlab # [ 293.241475] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:04.399Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"7ee2fb185d4d29e637ab7aa9","created_at":"2026-09-01T10:36:04.229Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f441dd7f7a124867628afc730b5d243","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-01T10:36:04.237Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-7ee2fb185d4d29e637ab7aa9: done: 0.060709 sec","job_status":"done","queue_duration_s":0.100707,"scheduling_latency_s":0.10074,"redis_calls":8,"redis_duration_s":0.006488,"redis_read_bytes":133,"redis_write_bytes":914,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000377,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":108,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003535,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002576,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.060709,"completed_at":"2026-09-01T10:36:04.399Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1640gitlab # [ 293.275562] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:04.406Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"0676b7df011fd706a77eb685","created_at":"2026-09-01T10:36:04.250Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"04838b7e8d05115a282f175f8bc142e6","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.254Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-0676b7df011fd706a77eb685: start","job_status":"start","queue_duration_s":0.152077,"scheduling_latency_s":0.152119}1641gitlab # [ 294.083748] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:05.240Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"0676b7df011fd706a77eb685","created_at":"2026-09-01T10:36:04.250Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"04838b7e8d05115a282f175f8bc142e6","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.254Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-0676b7df011fd706a77eb685: done: 0.834541 sec","job_status":"done","queue_duration_s":0.152077,"scheduling_latency_s":0.152119,"redis_calls":565,"redis_duration_s":0.412149,"redis_read_bytes":944,"redis_write_bytes":48975,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.002294,"redis_feature_flag_read_bytes":384,"redis_feature_flag_write_bytes":215,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.010713,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.399142,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.461902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.834541,"completed_at":"2026-09-01T10:36:05.240Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1642gitlab # [ 294.144651] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:05.302Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"ee8fced33860735aea128f46","created_at":"2026-09-01T10:36:04.260Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"df6df0ace6d1516e37fd844c0f1b8db9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.262Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-ee8fced33860735aea128f46: start","job_status":"start","queue_duration_s":1.039375,"scheduling_latency_s":1.039416}1643gitlab # [ 294.194949] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:05.351Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"ee8fced33860735aea128f46","created_at":"2026-09-01T10:36:04.260Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"df6df0ace6d1516e37fd844c0f1b8db9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.262Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-ee8fced33860735aea128f46: done: 0.049664 sec","job_status":"done","queue_duration_s":1.039375,"scheduling_latency_s":1.039416,"redis_calls":17,"redis_duration_s":0.0058130000000000005,"redis_read_bytes":1031,"redis_write_bytes":1638,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002853,"redis_feature_flag_read_bytes":1024,"redis_feature_flag_write_bytes":566,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001666,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001294,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000791442,"exclusive_lock_hold_duration_s":0.003535925,"duration_s":0.049664,"completed_at":"2026-09-01T10:36:05.351Z","load_balancing_strategy":"primary","db_duration_s":0.000782,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1644gitlab # [ 294.258529] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:05.416Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","jid":"ee90b3658359a7815ee0a08d","created_at":"2026-09-01T10:36:04.269Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"372fb482086ae38b8ae997191c80fc3a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:805014656d48c4962f83ad00758ac8cfab4de244cfbd1d771454e66bd9018911","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.337Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker JID-ee90b3658359a7815ee0a08d: start","job_status":"start","queue_duration_s":1.07902,"scheduling_latency_s":1.079059}1645gitlab # [ 294.326848] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:05.484Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","jid":"ee90b3658359a7815ee0a08d","created_at":"2026-09-01T10:36:04.269Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"372fb482086ae38b8ae997191c80fc3a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:805014656d48c4962f83ad00758ac8cfab4de244cfbd1d771454e66bd9018911","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.337Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker JID-ee90b3658359a7815ee0a08d: done: 0.068377 sec","job_status":"done","queue_duration_s":1.07902,"scheduling_latency_s":1.079059,"redis_calls":20,"redis_duration_s":0.006691,"redis_read_bytes":20,"redis_write_bytes":3248,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.003116,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1346,"redis_queues_calls":2,"redis_queues_duration_s":0.000369,"redis_queues_read_bytes":3,"redis_queues_write_bytes":818,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002195,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":734,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001011,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":350,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032629,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000395302,"duration_s":0.068377,"completed_at":"2026-09-01T10:36:05.484Z","load_balancing_strategy":"primary","db_duration_s":0.003556,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1646gitlab # [ 294.392072] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:05.549Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"5ef5f647f4e9f81c24c9ac7f","created_at":"2026-09-01T10:36:04.370Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.380Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-5ef5f647f4e9f81c24c9ac7f: start","job_status":"start","queue_duration_s":1.169631,"scheduling_latency_s":1.169666}1647gitlab # [ 294.536762] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:05.693Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"5ef5f647f4e9f81c24c9ac7f","created_at":"2026-09-01T10:36:04.370Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.380Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-5ef5f647f4e9f81c24c9ac7f: done: 0.143894 sec","job_status":"done","queue_duration_s":1.169631,"scheduling_latency_s":1.169666,"redis_calls":65,"redis_duration_s":0.023904,"redis_read_bytes":748,"redis_write_bytes":41402,"redis_feature_flag_calls":5,"redis_feature_flag_duration_s":0.00124,"redis_feature_flag_read_bytes":640,"redis_feature_flag_write_bytes":393,"redis_queues_calls":2,"redis_queues_duration_s":0.000999,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37385,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.020743,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000922,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.116375,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.143894,"completed_at":"2026-09-01T10:36:05.693Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1648gitlab # [ 294.598645] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:05.697Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"8cd0d49b979b4ead100d463f","created_at":"2026-09-01T10:36:04.394Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba36623b63613de715cb20b820d09d1a","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.403Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-8cd0d49b979b4ead100d463f: start","job_status":"start","queue_duration_s":1.293966,"scheduling_latency_s":1.293999}1649gitlab # [ 294.622614] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:05.730Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"8cd0d49b979b4ead100d463f","created_at":"2026-09-01T10:36:04.394Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba36623b63613de715cb20b820d09d1a","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.403Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-8cd0d49b979b4ead100d463f: done: 0.033513 sec","job_status":"done","queue_duration_s":1.293966,"scheduling_latency_s":1.293999,"redis_calls":11,"redis_duration_s":0.004172,"redis_read_bytes":517,"redis_write_bytes":1011,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001538,"redis_feature_flag_read_bytes":512,"redis_feature_flag_write_bytes":300,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001896,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000738,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021069,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033513,"completed_at":"2026-09-01T10:36:05.730Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003842,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1650gitlab # [ 294.682610] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:05.733Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"9a7cc0684fdb25ddd898dec1","created_at":"2026-09-01T10:36:04.414Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"58f128a7b455b3a086f8a9242acf6476","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.425Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-9a7cc0684fdb25ddd898dec1: start","job_status":"start","queue_duration_s":1.308101,"scheduling_latency_s":1.308135}1651gitlab # [ 294.702392] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:05.755Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"9a7cc0684fdb25ddd898dec1","created_at":"2026-09-01T10:36:04.414Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"58f128a7b455b3a086f8a9242acf6476","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.425Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-9a7cc0684fdb25ddd898dec1: done: 0.021971 sec","job_status":"done","queue_duration_s":1.308101,"scheduling_latency_s":1.308135,"redis_calls":11,"redis_duration_s":0.003271,"redis_read_bytes":138,"redis_write_bytes":877,"redis_cache_calls":3,"redis_cache_duration_s":0.00074,"redis_cache_read_bytes":5,"redis_cache_write_bytes":97,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000272,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":84,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001443,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000816,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013894,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021971,"completed_at":"2026-09-01T10:36:05.755Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1652gitlab # [ 294.704684] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:05.757Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"8d80dfff05b6640ae97f3afa","created_at":"2026-09-01T10:36:04.435Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"17cb4d24002492742ae086b6e9709959","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.441Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-8d80dfff05b6640ae97f3afa: start","job_status":"start","queue_duration_s":1.31551,"scheduling_latency_s":1.31554}1653gitlab # [ 294.780452] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:05.787Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"8d80dfff05b6640ae97f3afa","created_at":"2026-09-01T10:36:04.435Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"17cb4d24002492742ae086b6e9709959","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.441Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-8d80dfff05b6640ae97f3afa: done: 0.030392 sec","job_status":"done","queue_duration_s":1.31551,"scheduling_latency_s":1.31554,"redis_calls":8,"redis_duration_s":0.005206,"redis_read_bytes":133,"redis_write_bytes":936,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000377,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":110,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00374,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001089,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01728,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.030392,"completed_at":"2026-09-01T10:36:05.787Z","load_balancing_strategy":"primary","db_duration_s":0.002398,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1654gitlab # [ 294.782412] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:05.789Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"302cba0a4e7ee7246a27fbbf","created_at":"2026-09-01T10:36:04.454Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6216d932290932126fb42bd07e0e2c28","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.460Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-302cba0a4e7ee7246a27fbbf: start","job_status":"start","queue_duration_s":1.3291,"scheduling_latency_s":1.329131}1655gitlab # [ 294.803454] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:05.803Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1656gitlab # [ 294.808929] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:05.811Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"302cba0a4e7ee7246a27fbbf","created_at":"2026-09-01T10:36:04.454Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6216d932290932126fb42bd07e0e2c28","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.460Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-302cba0a4e7ee7246a27fbbf: done: 0.022191 sec","job_status":"done","queue_duration_s":1.3291,"scheduling_latency_s":1.329131,"redis_calls":8,"redis_duration_s":0.003118,"redis_read_bytes":133,"redis_write_bytes":937,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000273,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":111,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001866,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000979,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022191,"completed_at":"2026-09-01T10:36:05.811Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1657gitlab # [ 294.877119] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:05.815Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"b8059285bfe5980a3bf6da88","created_at":"2026-09-01T10:36:04.477Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"752c53c1aca1a501b30dd14698dd6f02","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.483Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-b8059285bfe5980a3bf6da88: start","job_status":"start","queue_duration_s":1.332163,"scheduling_latency_s":1.332201}1658gitlab # [ 294.891200] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:05.842Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"b8059285bfe5980a3bf6da88","created_at":"2026-09-01T10:36:04.477Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"752c53c1aca1a501b30dd14698dd6f02","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.483Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-b8059285bfe5980a3bf6da88: done: 0.026715 sec","job_status":"done","queue_duration_s":1.332163,"scheduling_latency_s":1.332201,"redis_calls":10,"redis_duration_s":0.004282,"redis_read_bytes":135,"redis_write_bytes":1184,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000316,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":97,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001912,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002054,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016311,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000770209,"exclusive_lock_hold_duration_s":0.003011835,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.026715,"completed_at":"2026-09-01T10:36:05.842Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000848,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1659gitlab # [ 294.969401] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:05.845Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"b6c2d38a39d31bdffb684cce","created_at":"2026-09-01T10:36:04.513Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6c48a19adbe7ea6dd0080676ad0297f0","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.520Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-b6c2d38a39d31bdffb684cce: start","job_status":"start","queue_duration_s":1.324961,"scheduling_latency_s":1.325}1660gitlab # [ 295.074772] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:06.230Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"b6c2d38a39d31bdffb684cce","created_at":"2026-09-01T10:36:04.513Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6c48a19adbe7ea6dd0080676ad0297f0","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.520Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-b6c2d38a39d31bdffb684cce: done: 0.384705 sec","job_status":"done","queue_duration_s":1.324961,"scheduling_latency_s":1.325,"redis_calls":9,"redis_duration_s":0.00916,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001333,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00647,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001357,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.135181,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.384705,"completed_at":"2026-09-01T10:36:06.229Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006114,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1661gitlab # [ 295.091459] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:06.245Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Runners::StaleMachinesCleanupCronWorker","jid":"a111a6d89a63c1a5599582ed","created_at":"2026-09-01T10:36:04.531Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3fff0d988eb09b802ba1da9084b16b6c","meta.root_caller_id":"Cronjob","meta.feature_category":"fleet_visibility","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b74a831dd4b14c3fc2b303407a0e209ea633ae1f03f57327e51b9e7be5871f1f","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.539Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Runners::StaleMachinesCleanupCronWorker JID-a111a6d89a63c1a5599582ed: start","job_status":"start","queue_duration_s":1.704877,"scheduling_latency_s":1.704952}1662gitlab # [ 295.165805] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:06.314Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Runners::StaleMachinesCleanupCronWorker","jid":"a111a6d89a63c1a5599582ed","created_at":"2026-09-01T10:36:04.531Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3fff0d988eb09b802ba1da9084b16b6c","meta.root_caller_id":"Cronjob","meta.feature_category":"fleet_visibility","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b74a831dd4b14c3fc2b303407a0e209ea633ae1f03f57327e51b9e7be5871f1f","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.539Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Runners::StaleMachinesCleanupCronWorker JID-a111a6d89a63c1a5599582ed: done: 0.070353 sec","job_status":"done","queue_duration_s":1.704877,"scheduling_latency_s":1.704952,"redis_calls":10,"redis_duration_s":0.005309,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000757,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00349,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001062,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000817422,"extra.ci_runners_stale_machines_cleanup_cron_worker.status":"success","extra.ci_runners_stale_machines_cleanup_cron_worker.total_deleted":0,"extra.ci_runners_stale_machines_cleanup_cron_worker.batch_counts":[0],"duration_s":0.070353,"completed_at":"2026-09-01T10:36:06.314Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019024,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1663gitlab # [ 295.236532] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:06.317Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"26be71a9d5be8cb012d493f3","created_at":"2026-09-01T10:36:04.550Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e81ebf3f852a8fc53ea1d1fb49b4034","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.556Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-26be71a9d5be8cb012d493f3: start","job_status":"start","queue_duration_s":1.761237,"scheduling_latency_s":1.761279}1664gitlab # [ 295.238354] setup-gitlab-start[2581]: + systemctl restart gitlab-sidekiq.service1665gitlab # [ 295.240268] runuser[2599]: pam_unix(runuser:session): session closed for user gitlab1666gitlab # [ 295.241841] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:06.337Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1667gitlab # [ 295.243609] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:06.340Z","message":"Shutting down"}1668gitlab # [ 295.244917] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:06.342Z","message":"Scheduler exiting..."}1669gitlab # [ 295.246079] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:06.342Z","message":"Terminating quiet threads for default capsule"}1670gitlab # [ 295.246994] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:06.343Z","message":"Scheduler exiting..."}1671gitlab # [ 295.247744] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:06.352Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"26be71a9d5be8cb012d493f3","created_at":"2026-09-01T10:36:04.550Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e81ebf3f852a8fc53ea1d1fb49b4034","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.556Z","job_size_bytes":2,"pid":1832,"sidekiq_tid":"manw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-26be71a9d5be8cb012d493f3: done: 0.035 sec","job_status":"done","queue_duration_s":1.761237,"scheduling_latency_s":1.761279,"redis_calls":9,"redis_duration_s":0.004929,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000729,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002579,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001621,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017486,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035,"completed_at":"2026-09-01T10:36:06.352Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000672,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1672gitlab # [ 295.343626] systemd[1]: Stopping gitlab-sidekiq.service...1673gitlab # [ 295.694962] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:06.852Z","message":"Bye!"}1674gitlab # [ 295.724711] sidekiq-cluster[1832]: {"severity":"INFO","time":"2026-09-01T10:36:06.882Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1832,"worker_id":"sidekiq_0","memwd_rss_bytes":965005312,"retry":0}1675gitlab # [ 299.775799] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1676gitlab # [ 299.779387] systemd[1]: Stopped gitlab-sidekiq.service.1677gitlab # [ 299.781738] systemd[1]: gitlab-sidekiq.service: Consumed 1min 33.445s CPU time over 3min 24.271s wall clock time, 972.6M memory peak, 4M read from disk.1678gitlab # [ 299.784187] systemd[1]: Started gitlab-sidekiq.service.1679gitlab # [ 299.785604] setup-gitlab-start[2581]: + mkdir -p /var/lib/secrets1680gitlab # [ 299.788376] setup-gitlab-start[2581]: + echo nixbot-test-token-of-20-chars1681gitlab # [ 299.789178] setup-gitlab-start[2581]: + chmod 644 /var/lib/secrets/gitlab-token1682gitlab # [ 299.791922] setup-gitlab-start[2581]: + curl -fs -X POST -H 'PRIVATE-TOKEN: nixbot-test-token-of-20-chars' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects1683gitlab # [ 300.988166] gitaly[916]: time="2026-09-01T10:36:12.145Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MVASVNWQYQKMYHM5YM7S error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:36:22.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:12.141" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1684gitlab # [ 300.989712] gitaly[916]: time="2026-09-01T10:36:12.147Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MVASVNWQYQKMYHM5YM7S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:36:22.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:12.141" grpc.time_ms=4.813 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1685gitlab # [ 301.179529] gitaly[916]: time="2026-09-01T10:36:12.337Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MVASVNWQYQKMYHM5YM7S error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:36:22.215" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:12.337" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1686gitlab # [ 301.196634] gitaly[916]: time="2026-09-01T10:36:12.354Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MVASVNWQYQKMYHM5YM7S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:36:22.215" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:12.337" grpc.time_ms=0.625 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1687gitlab # [ 301.257842] gitaly[916]: time="2026-09-01T10:36:12.415Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MVASVNWQYQKMYHM5YM7S diskcache=d0f3b17f-0d0f-4972-9270-5d58a9945bb6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-01T10:36:22.241" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:12.362" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1688gitlab # [ 301.274786] gitaly[916]: time="2026-09-01T10:36:12.432Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1572 command.oublock=32 command.real_time_ms=17 command.spawn_token_fork_ms=4 command.system_time_ms=8 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MVASVNWQYQKMYHM5YM7S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-01T10:36:22.241" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:12.362" grpc.time_ms=53.279 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1689gitlab # [ 301.293109] gitaly[916]: time="2026-09-01T10:36:12.450Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MVASVNWQYQKMYHM5YM7S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:36:22.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:12.450" pid=916 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1690gitlab # [ 301.295322] gitaly[916]: time="2026-09-01T10:36:12.451Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MVASVNWQYQKMYHM5YM7S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:36:22.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:12.450" grpc.time_ms=1.181 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1691gitlab # [ 301.313664] gitaly[916]: time="2026-09-01T10:36:12.471Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MVASVNWQYQKMYHM5YM7S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-01T10:36:22.022" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:12.462" grpc.time_ms=8.226 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1692gitlab # [ 301.403092] gitaly[916]: time="2026-09-01T10:36:12.561Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MVASVNWQYQKMYHM5YM7S error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:36:22.119" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:12.560" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1693gitlab # [ 301.405128] gitaly[916]: time="2026-09-01T10:36:12.563Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MVASVNWQYQKMYHM5YM7S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:36:22.119" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:12.560" grpc.time_ms=0.511 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1694gitlab # [ 301.498227] gitaly[916]: time="2026-09-01T10:36:12.656Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MVASVNWQYQKMYHM5YM7S diskcache=b53a9f27-6104-43df-b372-4febf2ccb505 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-01T10:36:22.156" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:12.597" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1695gitlab # [ 301.498949] gitaly[916]: time="2026-09-01T10:36:12.656Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1581 command.oublock=32 command.real_time_ms=19 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MVASVNWQYQKMYHM5YM7S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-01T10:36:22.156" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:12.597" grpc.time_ms=58.736 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1696gitlab # [ 301.545216] gitaly[916]: time="2026-09-01T10:36:12.703Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MVASVNWQYQKMYHM5YM7S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:36:22.261" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:12.702" pid=916 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1697gitlab # [ 301.561402] gitaly[916]: time="2026-09-01T10:36:12.703Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MVASVNWQYQKMYHM5YM7S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:36:22.261" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:12.702" grpc.time_ms=1.262 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1698gitlab # [ 301.588296] gitaly[916]: time="2026-09-01T10:36:12.746Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MVASVNWQYQKMYHM5YM7S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-01T10:36:22.279" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:12.736" grpc.time_ms=9.315 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1699gitlab # [ 301.630773] gitaly[916]: time="2026-09-01T10:36:12.788Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MVASVNWQYQKMYHM5YM7S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:36:22.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:12.787" grpc.time_ms=0.436 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1700gitlab # [ 301.644699] gitaly[916]: time="2026-09-01T10:36:12.802Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MVASVNWQYQKMYHM5YM7S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-01T10:36:22.019" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:12.800" grpc.time_ms=0.636 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1701gitlab # [ 302.585119] gitaly[916]: time="2026-09-01T10:36:13.742Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MVASVNWQYQKMYHM5YM7S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-01T10:36:23.290" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:13.732" grpc.time_ms=9.955 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1702gitlab # [ 303.049987] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:14 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 32361703gitlab # [ 303.053399] setup-gitlab-start[2581]: + mkdir -p /tmp/test-flake1704gitlab # [ 303.054922] setup-gitlab-start[2581]: + cd /tmp/test-flake1705gitlab # [ 303.056400] setup-gitlab-start[2581]: + git init -b master1706gitlab # [ 303.072334] setup-gitlab-start[3125]: Initialized empty Git repository in /tmp/test-flake/.git/1707gitlab # [ 303.074387] setup-gitlab-start[2581]: + git config user.name test1708gitlab # [ 303.075676] setup-gitlab-start[2581]: + git config user.email test@example.com1709gitlab # [ 303.077446] setup-gitlab-start[2581]: + cat1710gitlab # [ 303.078552] setup-gitlab-start[2581]: + git add flake.nix1711gitlab # [ 303.080449] setup-gitlab-start[2581]: + git commit -m 'initial commit'1712gitlab # [ 303.087392] setup-gitlab-start[3130]: [master (root-commit) ae86dcc] initial commit1713gitlab # [ 303.089439] setup-gitlab-start[3130]: 1 file changed, 10 insertions(+)1714gitlab # [ 303.091219] setup-gitlab-start[3130]: create mode 100644 flake.nix1715gitlab # [ 303.092841] setup-gitlab-start[2581]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1716gitlab # [ 303.095245] setup-gitlab-start[2581]: + git push -u origin master1717gitlab # [ 303.187528] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:14 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 731718gitlab # [ 303.616122] gitaly[916]: time="2026-09-01T10:36:14.773Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=4 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8MYMBCSKZGB3V04S1G7RK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:14.746" grpc.time_ms=23.989 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1719gitlab # [ 303.655217] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:14 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 4261720gitlab # [ 305.156596] gitaly[916]: time="2026-09-01T10:36:16.314Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-01T10:36:26.182" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:16.303" grpc.time_ms=10.01 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1721gitlab # [ 305.262172] gitaly[916]: time="2026-09-01T10:36:16.419Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=8 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-01T10:36:46.097" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-01T10:36:16.398" grpc.time_ms=19.871 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1722gitlab # [ 305.344607] gitaly[916]: time="2026-09-01T10:36:16.502Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K duration_ms=337ns method=POST pid=916 status=200 url="http://unix/api/v4/internal/allowed"1723gitlab # [ 305.637279] gitaly[916]: time="2026-09-01T10:36:16.795Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K duration_ms=285ns method=POST pid=916 status=200 url="http://unix/api/v4/internal/pre_receive"1724gitlab # [ 305.643727] gitaly[916]: time="2026-09-01T10:36:16.801Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:16.152" grpc.time_ms=648.845 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1725gitlab # [ 305.701324] gitaly[916]: time="2026-09-01T10:36:16.858Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:16.855" grpc.time_ms=2.466 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1726gitlab # [ 305.956391] gitaly[916]: time="2026-09-01T10:36:17.114Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:17.112" grpc.time_ms=1.675 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1727gitlab # [ 306.035617] gitaly[916]: time="2026-09-01T10:36:17.192Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:17.189" grpc.time_ms=2.605 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1728gitlab # [ 306.265575] gitaly[916]: time="2026-09-01T10:36:17.421Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-01T10:36:27.287" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:17.408" grpc.time_ms=12.226 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1729gitlab # [ 306.325657] gitaly[916]: time="2026-09-01T10:36:17.483Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1573 command.oublock=0 command.real_time_ms=26 command.spawn_token_fork_ms=12 command.system_time_ms=8 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-01T10:36:27.002" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:36:17.443" grpc.time_ms=39.374 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1730gitlab # [ 306.380686] gitaly[916]: time="2026-09-01T10:36:17.538Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K duration_ms=235ns method=POST pid=916 status=200 url="http://unix/api/v4/internal/post_receive"1731gitlab # [ 306.384182] gitaly[916]: time="2026-09-01T10:36:17.541Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:17.301" grpc.time_ms=240.21 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1732gitlab # [ 306.423665] gitaly[916]: time="2026-09-01T10:36:17.580Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K diskcache=3bd9cfb9-a5fa-416e-bd7a-56f0a4b8cb22 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:15.928" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1733gitlab # [ 306.424251] gitaly[916]: time="2026-09-01T10:36:17.580Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=289 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13458 command.oublock=32 command.real_time_ms=1629 command.spawn_token_fork_ms=5 command.system_time_ms=162 command.user_time_ms=127 component=gitaly.StreamServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:15.928" grpc.time_ms=1652.345 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1734gitlab # [ 306.425817] gitlab-workhorse[826]: time="2026-09-01T10:36:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521735gitlab # [ 306.428108] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:17 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 27521736gitlab # [ 306.559093] setup-gitlab-start[3134]: To http://gitlab/root/test-flake.git1737gitlab # [ 306.559621] setup-gitlab-start[3134]: * [new branch] master -> master1738gitlab # [ 306.590804] setup-gitlab-start[3134]: branch 'master' set up to track 'origin/master'.1739gitlab # [ 306.601866] systemd[1]: Finished setup-gitlab.service.1740gitlab # [ 306.616804] systemd[1]: Starting nixbot CI service...1741gitlab: (finished: waiting for unit setup-gitlab.service, in 308.00 seconds)1742gitlab: waiting for unit nixbot.service1743gitlab # [ 308.042113] sidekiq-cluster[3074]: {"severity":"INFO","time":"2026-09-01T10:36:19.100Z","message":"Starting cluster with 1 processes"}1744gitlab # [ 308.849575] redis[789]: 10 changes in 300 seconds. Saving...1745gitlab # [ 308.849968] redis[789]: Background saving started by pid 32501746gitlab # [ 308.863407] redis[3250]: BGSAVE done, 488 keys saved, 0 keys skipped, 125953 bytes written.1747gitlab # [ 308.885631] redis[3250]: DB saved on disk1748gitlab # [ 308.887500] redis[3250]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB1749gitlab # [ 308.955657] redis[789]: Background saving terminated with success1750gitlab # [ 308.982300] postgres[866]: [866] LOG: checkpoint starting: time1751gitlab # [ 311.047508] gitaly[916]: time="2026-09-01T10:36:22.204Z" level=info msg="burden monitor stats" pid=9161752gitlab # [ 311.841568] awbbx6p3slpsgq98a2l19krl2hhkd5pf-nixbot-health[3236]: curl: (28) Operation timed out after 5002 milliseconds with 0 bytes received1753gitlab # [ 312.219636] nixbot[3225]: {"ts": "2026-09-01T10:36:23.377Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1754gitlab # [ 312.258408] nixbot[3225]: {"ts": "2026-09-01T10:36:23.416Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1755gitlab # [ 312.287111] nixbot[3225]: {"ts": "2026-09-01T10:36:23.444Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1756gitlab # [ 312.296148] nixbot[3225]: {"ts": "2026-09-01T10:36:23.453Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1757gitlab # [ 312.304686] nixbot[3225]: {"ts": "2026-09-01T10:36:23.462Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1758gitlab # [ 312.312460] nixbot[3225]: {"ts": "2026-09-01T10:36:23.470Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1759gitlab # [ 312.317362] nixbot[3225]: {"ts": "2026-09-01T10:36:23.475Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1760gitlab # [ 312.324296] nixbot[3225]: {"ts": "2026-09-01T10:36:23.478Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1761gitlab # [ 312.329385] nixbot[3225]: {"ts": "2026-09-01T10:36:23.487Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1762gitlab # [ 312.336192] nixbot[3225]: {"ts": "2026-09-01T10:36:23.494Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1763gitlab # [ 312.341358] nixbot[3225]: {"ts": "2026-09-01T10:36:23.499Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1764gitlab # [ 312.353908] nixbot[3225]: {"ts": "2026-09-01T10:36:23.511Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1765gitlab # [ 312.358636] nixbot[3225]: {"ts": "2026-09-01T10:36:23.516Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1766gitlab # [ 312.364955] nixbot[3225]: {"ts": "2026-09-01T10:36:23.522Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1767gitlab # [ 312.370725] nixbot[3225]: {"ts": "2026-09-01T10:36:23.528Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1768gitlab # [ 312.376067] nixbot[3225]: {"ts": "2026-09-01T10:36:23.534Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1769gitlab # [ 312.383240] nixbot[3225]: {"ts": "2026-09-01T10:36:23.541Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1770gitlab # [ 312.388304] nixbot[3225]: {"ts": "2026-09-01T10:36:23.546Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1771gitlab # [ 312.397336] nixbot[3225]: {"ts": "2026-09-01T10:36:23.555Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1772gitlab # [ 312.399892] nixbot[3225]: {"ts": "2026-09-01T10:36:23.557Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1773gitlab # [ 312.406095] nixbot[3225]: {"ts": "2026-09-01T10:36:23.564Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1774gitlab # [ 312.414589] nixbot[3225]: {"ts": "2026-09-01T10:36:23.572Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1775gitlab # [ 312.420843] nixbot[3225]: {"ts": "2026-09-01T10:36:23.578Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1776gitlab # [ 312.424566] nixbot[3225]: {"ts": "2026-09-01T10:36:23.582Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1777gitlab # [ 312.427084] nixbot[3225]: {"ts": "2026-09-01T10:36:23.585Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1778gitlab # [ 312.431698] nixbot[3225]: {"ts": "2026-09-01T10:36:23.589Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1779gitlab # [ 312.434961] nixbot[3225]: {"ts": "2026-09-01T10:36:23.592Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1780gitlab # [ 312.437628] nixbot[3225]: {"ts": "2026-09-01T10:36:23.595Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1781gitlab # [ 312.825297] nixbot[3225]: INFO: Started server process [3225]1782gitlab # [ 312.825907] nixbot[3225]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1783gitlab # [ 312.861855] nixbot[3225]: INFO: ::ffff:127.0.0.1:48712 - "GET /health HTTP/1.1" 200 OK1784gitlab # [ 312.867974] systemd[1]: Started nixbot CI service.1785gitlab # [ 312.870258] systemd[1]: Reached target Multi-User System.1786gitlab # [ 312.872184] systemd[1]: Startup finished in 1.302s (kernel) + 3.502s (initrd) + 5min 8.066s (userspace) = 5min 12.871s.1787gitlab: (finished: waiting for unit nixbot.service, in 6.37 seconds)1788gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1789gitlab # [ 313.470978] nixbot[3225]: INFO: ::ffff:127.0.0.1:49842 - "GET /health HTTP/1.1" 200 OK1790gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.02 seconds)1791subtest: project discovered and webhook registered1792gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1793gitlab # [ 313.554046] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:24 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 631794gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.09 seconds)1795[]1796gitlab # [ 313.816208] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:24 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 9571797gitlab # [ 313.820333] nixbot[3225]: {"ts": "2026-09-01T10:36:24.975Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1798gitlab # [ 313.836398] nixbot[3225]: {"ts": "2026-09-01T10:36:24.994Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1799gitlab # [ 313.890960] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:25 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 471800gitlab # [ 313.895287] nixbot[3225]: {"ts": "2026-09-01T10:36:25.053Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1801gitlab # [ 313.898526] nixbot[3225]: {"ts": "2026-09-01T10:36:25.054Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1802gitlab # [ 314.058961] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:25 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 919 "" "python-httpx/0.28.1" 1591803gitlab # [ 314.062356] nixbot[3225]: {"ts": "2026-09-01T10:36:25.220Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1804gitlab # [ 314.197413] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:25 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 1251805gitlab # [ 314.200983] nixbot[3225]: {"ts": "2026-09-01T10:36:25.359Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1806gitlab # [ 314.304925] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:25 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 882 "" "python-httpx/0.28.1" 931807gitlab # [ 314.308626] nixbot[3225]: {"ts": "2026-09-01T10:36:25.466Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1808gitlab # [ 314.370892] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:25 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 919 "" "python-httpx/0.28.1" 551809gitlab # [ 314.375782] nixbot[3225]: {"ts": "2026-09-01T10:36:25.529Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1810gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1811gitlab # [ 314.540616] gitaly[916]: time="2026-09-01T10:36:25.698Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8N9J3W8V4DM7HV3WDANC1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-01T10:36:55.385" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:36:25.687" grpc.time_ms=8.974 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1812gitlab # [ 314.579932] gitaly[916]: time="2026-09-01T10:36:25.737Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8N9J3W8V4DM7HV3WDANC1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:36:55.435" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:36:25.736" grpc.time_ms=1.217 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1813gitlab # [ 314.609094] gitaly[916]: time="2026-09-01T10:36:25.767Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8N9J3W8V4DM7HV3WDANC1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-01T10:36:35.008" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:36:25.758" grpc.time_ms=7.694 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1814gitlab # [ 314.630746] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:25 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 882 "" "curl/8.21.0" 501815gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.08 seconds)1816[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-01T10:36:25.200Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]1817(finished: subtest: project discovered and webhook registered, in 1.16 seconds)1818subtest: push delivers the webhook1819gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1820gitlab # [ 314.646284] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:25 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 2641821gitlab # [ 314.650515] nixbot[3225]: {"ts": "2026-09-01T10:36:25.808Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1822gitlab # [ 314.707116] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:25 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 401823gitlab # [ 314.751143] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:25 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 961824gitlab # [ 314.755282] nixbot[3225]: {"ts": "2026-09-01T10:36:25.913Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1825gitlab # [ 314.763109] nixbot[3225]: {"ts": "2026-09-01T10:36:25.920Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1826gitlab # [ 314.812812] gitaly[916]: time="2026-09-01T10:36:25.970Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8N9WFH9WZMDZ21C46FAGP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:25.958" grpc.time_ms=11.56 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1827gitlab # [ 314.833652] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:25 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 201 "" "git/2.55.0" 1181828gitlab # Enumerating objects: 5, done.1829gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1830gitlab # Delta compression using up to 4 threads1831gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1832gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 274.00 KiB/s, done.1833gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1834gitlab # [ 314.857054] gitaly[916]: time="2026-09-01T10:36:26.015Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8N9YTGGTSG85KB390EX3W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:25.995" grpc.time_ms=19.549 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1835gitlab # [ 314.874561] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:26 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 691836gitlab # [ 314.959785] gitaly[916]: time="2026-09-01T10:36:26.117Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NA11DKKJCHJ9TJN8X6V4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:26.102" pid=916 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1837gitlab # [ 314.975449] gitaly[916]: time="2026-09-01T10:36:26.117Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NA11DKKJCHJ9TJN8X6V4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:26.102" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1838gitlab # [ 314.993471] gitlab-workhorse[826]: time="2026-09-01T10:36:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131839gitlab # [ 314.996102] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:26 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 1111840gitlab # [ 314.998748] gitaly[916]: time="2026-09-01T10:36:26.121Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NA11DKKJCHJ9TJN8X6V4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:26.102" grpc.time_ms=17.683 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1841gitlab # [ 315.067413] gitaly[916]: time="2026-09-01T10:36:26.225Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NA51SS0K2DJ7JYNX0X3K grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:26.209" pid=916 remote_ip="2001:db8:1::1" request_sha=0876f75eadc8fff4749bba0b2ec7dcfef2c0eb17 response_bytes=228 span.kind=server system=grpc1842gitlab # [ 315.084179] gitaly[916]: time="2026-09-01T10:36:26.225Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1211 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NA51SS0K2DJ7JYNX0X3K grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:26.209" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1843gitlab # [ 315.102160] gitlab-workhorse[826]: time="2026-09-01T10:36:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2281844gitlab # [ 315.102808] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:26 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 228 "" "git/2.55.0" 811845gitlab # [ 315.103761] gitaly[916]: time="2026-09-01T10:36:26.238Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1211 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NA51SS0K2DJ7JYNX0X3K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:26.209" grpc.time_ms=16.567 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1846gitlab # [ 315.104669] gitaly[916]: time="2026-09-01T10:36:26.249Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-01T10:36:56.041" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-01T10:36:26.242" grpc.time_ms=7.359 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1847gitlab # [ 315.149214] gitaly[916]: time="2026-09-01T10:36:26.306Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K duration_ms=168ns method=POST pid=916 status=200 url="http://unix/api/v4/internal/allowed"1848gitlab # [ 315.175338] gitaly[916]: time="2026-09-01T10:36:26.333Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K duration_ms=25ns method=POST pid=916 status=200 url="http://unix/api/v4/internal/pre_receive"1849gitlab # [ 315.180063] gitaly[916]: time="2026-09-01T10:36:26.333Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:26.137" grpc.time_ms=196.057 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1850gitlab # [ 315.218054] gitaly[916]: time="2026-09-01T10:36:26.376Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:26.374" grpc.time_ms=1.381 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1851gitlab # [ 315.256686] gitaly[916]: time="2026-09-01T10:36:26.414Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NAAJ5XAQ21ZE4G39ZB3A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:26.401" grpc.time_ms=12.448 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1852gitlab # [ 315.272703] gitaly[916]: time="2026-09-01T10:36:26.415Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:26.415" grpc.time_ms=0.486 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1853gitlab # [ 315.285860] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:26 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 931854gitlab # [ 315.292977] gitaly[916]: time="2026-09-01T10:36:26.451Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:26.450" grpc.time_ms=0.555 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1855gitlab # [ 315.342682] gitaly[916]: time="2026-09-01T10:36:26.500Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NADKZ2ERQV558EB9EQWT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:26.492" pid=916 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1856gitlab # [ 315.355920] gitaly[916]: time="2026-09-01T10:36:26.500Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NADKZ2ERQV558EB9EQWT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:26.492" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1857gitlab # [ 315.372618] gitlab-workhorse[826]: time="2026-09-01T10:36:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671858gitlab # [ 315.375261] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:26 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 821859gitlab # [ 315.377793] gitaly[916]: time="2026-09-01T10:36:26.501Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NADKZ2ERQV558EB9EQWT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:26.492" grpc.time_ms=8.843 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1860gitlab # [ 315.409997] gitaly[916]: time="2026-09-01T10:36:26.568Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NAG8ANV4SX7YKRG71JTA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:26.551" pid=916 remote_ip="2001:db8:1::1" request_sha=1907a9a26ba312a17ad39be4048d9c0e570f3011 response_bytes=336 span.kind=server system=grpc1861gitlab # [ 315.423265] gitaly[916]: time="2026-09-01T10:36:26.568Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1197 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NAG8ANV4SX7YKRG71JTA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:26.551" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1862gitlab # [ 315.440411] gitlab-workhorse[826]: time="2026-09-01T10:36:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3361863gitlab # [ 315.443783] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:26 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 336 "" "git/2.55.0" 651864gitlab # [ 315.444628] gitaly[916]: time="2026-09-01T10:36:26.569Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1197 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NAG8ANV4SX7YKRG71JTA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:26.551" grpc.time_ms=17.264 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1865gitlab # [ 315.447855] gitaly[916]: time="2026-09-01T10:36:26.606Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K duration_ms=126ns method=POST pid=916 status=200 url="http://unix/api/v4/internal/post_receive"1866gitlab # [ 315.449986] gitaly[916]: time="2026-09-01T10:36:26.608Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:26.479" grpc.time_ms=128.409 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1867gitlab # [ 315.487245] gitaly[916]: time="2026-09-01T10:36:26.645Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K diskcache=05574a77-86a4-4187-bf25-4eff5fa32c4c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:26.083" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1868gitlab # To http://gitlab/root/test-flake.git1869gitlab # [ 315.490178] gitlab-workhorse[826]: time="2026-09-01T10:36:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521870gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.87 seconds)1871gitlab # ae86dcc..d996dce [ 315.491749] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:26 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 6351872gitlab: must succeed: git -C /tmp/test-flake rev-parse master1873gitlab # master -> master1874gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)1875gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'1876gitlab # [ 315.493990] gitaly[916]: time="2026-09-01T10:36:26.647Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=161 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13401 command.oublock=32 command.real_time_ms=555 command.spawn_token_fork_ms=2 command.system_time_ms=108 command.user_time_ms=53 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=58 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:26.083" grpc.time_ms=563.218 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1877gitlab # [ 315.564510] gitaly[916]: time="2026-09-01T10:36:26.722Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NANF7QJXTKFS9FS2AAAK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:26.716" grpc.time_ms=5.282 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1878gitlab # [ 315.570813] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:26 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 531879gitlab # [ 315.591212] nixbot[3225]: INFO: ::ffff:127.0.0.1:49848 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK1880gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.08 seconds)1881[]1882gitlab # [ 315.640890] gitaly[916]: time="2026-09-01T10:36:26.798Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NAQ7844HT7QZWSAE43M4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:26.789" pid=916 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1883gitlab # [ 315.643978] gitaly[916]: time="2026-09-01T10:36:26.798Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1174 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NAQ7844HT7QZWSAE43M4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:26.789" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1884gitlab # [ 315.645623] gitlab-workhorse[826]: time="2026-09-01T10:36:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091885gitlab # [ 315.646740] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:26 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 721886gitlab # [ 315.647917] gitaly[916]: time="2026-09-01T10:36:26.798Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1174 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NAQ7844HT7QZWSAE43M4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:26.789" grpc.time_ms=9.271 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1887gitlab # [ 315.777066] gitaly[916]: time="2026-09-01T10:36:26.935Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:36:56.733" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:36:26.934" grpc.time_ms=0.518 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1888gitlab # [ 315.827527] gitaly[916]: time="2026-09-01T10:36:26.985Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-01T10:36:56.779" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:36:26.980" grpc.time_ms=5.07 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1889gitlab # [ 315.863562] gitaly[916]: time="2026-09-01T10:36:27.021Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:36:37.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:36:27.016" grpc.time_ms=5.134 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1890gitlab # [ 315.897070] gitaly[916]: time="2026-09-01T10:36:27.054Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:36:37.036" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:36:27.046" grpc.time_ms=6.119 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1891gitlab # [ 315.985417] gitaly[916]: time="2026-09-01T10:36:27.143Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-01T10:36:37.132" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:27.142" grpc.time_ms=0.341 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1892gitlab # [ 316.012178] gitaly[916]: time="2026-09-01T10:36:27.170Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-01T10:36:37.154" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:36:27.164" grpc.time_ms=5.38 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1893gitlab # [ 316.060716] gitaly[916]: time="2026-09-01T10:36:27.218Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:27.216" grpc.time_ms=1.544 pid=916 span.kind=server system=grpc1894gitlab # [ 316.092383] gitaly[916]: time="2026-09-01T10:36:27.250Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:27.249" grpc.time_ms=0.491 pid=916 span.kind=server system=grpc1895gitlab # [ 316.109423] gitaly[916]: time="2026-09-01T10:36:27.267Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS diskcache=c5d83a0b-801c-4455-8ea5-dda2e48c1708 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-01T10:36:37.179" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:27.190" pid=916 ref=refs/keep-around/ae86dcce1d51f83b4a05ba73d940f28faca8fab9 remote_ip="2001:db8:1::1" revision=ae86dcce1d51f83b4a05ba73d940f28faca8fab9 span.kind=server system=grpc username=root1896gitlab # [ 316.109973] gitaly[916]: time="2026-09-01T10:36:27.267Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=55 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5577 command.oublock=8 command.real_time_ms=70 command.spawn_token_fork_ms=0 command.system_time_ms=30 command.user_time_ms=25 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-01T10:36:37.179" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:27.190" grpc.time_ms=77.439 pid=916 ref=refs/keep-around/ae86dcce1d51f83b4a05ba73d940f28faca8fab9 remote_ip="2001:db8:1::1" revision=ae86dcce1d51f83b4a05ba73d940f28faca8fab9 span.kind=server system=grpc username=root1897gitlab # [ 316.114190] gitaly[916]: time="2026-09-01T10:36:27.272Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:36:57.071" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:36:27.271" grpc.time_ms=0.326 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1898gitlab # [ 316.253364] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:27 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ae86dcce1d51f83b4a05ba73d940f28faca8fab9 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 5361899gitlab # [ 316.254858] nixbot[3225]: {"ts": "2026-09-01T10:36:27.412Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ae86dcce1d51f83b4a05ba73d940f28faca8fab9 \"HTTP/1.1 201 Created\""}1900gitlab # [ 316.261705] nixbot[3225]: {"ts": "2026-09-01T10:36:27.419Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-ae86dcce1d51-0"}1901gitlab # [ 316.431324] nixbot[3225]: {"ts": "2026-09-01T10:36:27.589Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-ae86dcce1d51-0", "argv0": "bwrap"}1902gitlab # [ 316.505436] systemd[1]: Started Nix Daemon.1903gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'1904gitlab # [ 316.589139] nix-daemon[3485]: accepted connection from pid 3480, user nixbot1905gitlab # [ 316.697052] nixbot[3225]: INFO: ::ffff:127.0.0.1:49858 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK1906gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.09 seconds)1907[]1908gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'1909gitlab # [ 317.720712] nixbot[3225]: INFO: ::ffff:127.0.0.1:49872 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK1910gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)1911[]1912gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'1913gitlab # [ 318.742612] nixbot[3225]: INFO: ::ffff:127.0.0.1:49876 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK1914gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)1915[]1916gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'1917gitlab # [ 319.766390] nixbot[3225]: INFO: ::ffff:127.0.0.1:49884 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK1918gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)1919[]1920gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'1921gitlab # [ 320.789401] nixbot[3225]: INFO: ::ffff:127.0.0.1:49900 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK1922gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)1923[]1924gitlab # [ 321.037182] nix-daemon[3485]: accepted connection from pid 3511, user nixbot1925gitlab # [ 321.052940] nix-daemon[3485]: accepted connection from pid 3480, user nixbot1926gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'1927gitlab # [ 321.810730] nixbot[3225]: INFO: ::ffff:127.0.0.1:49908 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK1928gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)1929[]1930gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'1931gitlab # [ 322.854588] nixbot[3225]: INFO: ::ffff:127.0.0.1:49924 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK1932gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.05 seconds)1933[]1934gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'1935gitlab # [ 323.878046] nixbot[3225]: INFO: ::ffff:127.0.0.1:60014 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK1936gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)1937[]1938gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'1939gitlab # [ 324.899098] nixbot[3225]: INFO: ::ffff:127.0.0.1:60016 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK1940gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)1941[]1942gitlab # [ 325.575983] nix-daemon[3485]: accepted connection from pid 3480, user nixbot1943gitlab # [ 325.629439] nix-daemon[3485]: accepted connection from pid 3541, user nixbot1944gitlab # [ 325.663658] gitaly[916]: time="2026-09-01T10:36:36.821Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NMH6VD9EAKQTM2JDZHJM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:37:06.619" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:36:36.820" grpc.time_ms=1.028 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1945gitlab # [ 325.713742] gitaly[916]: time="2026-09-01T10:36:36.871Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NMH6VD9EAKQTM2JDZHJM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:36:46.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:36:36.866" grpc.time_ms=5.425 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1946gitlab # [ 325.757605] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:36 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ae86dcce1d51f83b4a05ba73d940f28faca8fab9 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 1401947gitlab # [ 325.758293] nixbot[3225]: {"ts": "2026-09-01T10:36:36.916Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ae86dcce1d51f83b4a05ba73d940f28faca8fab9 \"HTTP/1.1 201 Created\""}1948gitlab # [ 325.819961] gitaly[916]: time="2026-09-01T10:36:36.978Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NMNVSYGNY0BNM3BRE0KS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:37:06.676" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:36:36.977" grpc.time_ms=0.724 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1949gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'1950gitlab # [ 325.923678] gitaly[916]: time="2026-09-01T10:36:37.081Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NMNVSYGNY0BNM3BRE0KS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:36:47.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:36:37.075" grpc.time_ms=6.111 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1951gitlab # [ 325.944919] nixbot[3225]: INFO: ::ffff:127.0.0.1:60024 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK1952gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.05 seconds)1953[]1954gitlab # [ 326.042976] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:37 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ae86dcce1d51f83b4a05ba73d940f28faca8fab9 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2761955gitlab # [ 326.047167] nixbot[3225]: {"ts": "2026-09-01T10:36:37.202Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ae86dcce1d51f83b4a05ba73d940f28faca8fab9 \"HTTP/1.1 201 Created\""}1956gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'1957gitlab # [ 326.970511] nixbot[3225]: INFO: ::ffff:127.0.0.1:60028 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK1958gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)1959[]1960gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'1961gitlab # [ 327.992808] nixbot[3225]: INFO: ::ffff:127.0.0.1:60034 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK1962gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)1963[]1964gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'1965gitlab # [ 329.014701] nixbot[3225]: INFO: ::ffff:127.0.0.1:60042 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK1966gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)1967[]1968gitlab # [ 329.976643] nix-daemon[3485]: accepted connection from pid 3584, user nixbot1969gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'1970gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.03 seconds)1971[]1972gitlab # [ 330.044077] nixbot[3225]: INFO: ::ffff:127.0.0.1:60050 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK1973gitlab # [ 330.090114] gitaly[916]: time="2026-09-01T10:36:41.247Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NRTR22KNQW9215XP9QNT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:37:11.045" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:36:41.246" grpc.time_ms=0.727 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1974gitlab # [ 330.132824] gitaly[916]: time="2026-09-01T10:36:41.290Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NRTR22KNQW9215XP9QNT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:36:51.213" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:36:41.284" grpc.time_ms=6.75 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1975gitlab # [ 330.211676] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:41 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ae86dcce1d51f83b4a05ba73d940f28faca8fab9 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1911976gitlab # [ 330.212339] nixbot[3225]: {"ts": "2026-09-01T10:36:41.370Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ae86dcce1d51f83b4a05ba73d940f28faca8fab9 \"HTTP/1.1 201 Created\""}1977gitlab # [ 330.416665] gitaly[916]: time="2026-09-01T10:36:41.574Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NS52K6V16X70PHC0AH9J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:41.567" grpc.time_ms=7.058 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1978gitlab # [ 330.435217] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:41 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 691979gitlab # [ 330.490623] gitaly[916]: time="2026-09-01T10:36:41.648Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NS79B8XEPPMZ11DCQSDT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:41.640" pid=916 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1980gitlab # [ 330.491489] gitaly[916]: time="2026-09-01T10:36:41.648Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NS79B8XEPPMZ11DCQSDT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:41.640" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1981gitlab # [ 330.493995] gitaly[916]: time="2026-09-01T10:36:41.649Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NS79B8XEPPMZ11DCQSDT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:41.640" grpc.time_ms=8.871 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1982gitlab # [ 330.494993] gitlab-workhorse[826]: time="2026-09-01T10:36:41Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671983gitlab # [ 330.495924] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:41 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 731984gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'1985gitlab # [ 331.071126] nixbot[3225]: INFO: ::ffff:127.0.0.1:60066 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK1986gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.03 seconds)1987[]1988gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'1989gitlab # [ 332.097719] nixbot[3225]: INFO: ::ffff:127.0.0.1:60072 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK1990gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)1991[]1992gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'1993gitlab # [ 333.125644] nixbot[3225]: INFO: ::ffff:127.0.0.1:43180 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK1994gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.03 seconds)1995[]1996gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'1997gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.05 seconds)1998[]1999gitlab # [ 334.180845] nixbot[3225]: INFO: ::ffff:127.0.0.1:43190 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2000gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2001gitlab # [ 335.209876] nixbot[3225]: INFO: ::ffff:127.0.0.1:43206 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2002gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.03 seconds)2003[]2004gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2005gitlab # [ 336.238064] nixbot[3225]: INFO: ::ffff:127.0.0.1:43216 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2006gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.03 seconds)2007[]2008gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2009gitlab # [ 337.287502] nixbot[3225]: INFO: ::ffff:127.0.0.1:43226 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2010gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.05 seconds)2011[]2012gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2013gitlab # [ 338.341243] nixbot[3225]: INFO: ::ffff:127.0.0.1:43230 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2014gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.05 seconds)2015[]2016gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2017gitlab # [ 339.374690] nixbot[3225]: INFO: ::ffff:127.0.0.1:43246 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2018gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.03 seconds)2019[]2020gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2021gitlab # [ 340.423130] nixbot[3225]: INFO: ::ffff:127.0.0.1:43248 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2022gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.05 seconds)2023[]2024gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2025gitlab # [ 341.472396] nixbot[3225]: INFO: ::ffff:127.0.0.1:43252 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2026gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.05 seconds)2027[]2028gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2029gitlab # [ 342.509138] nixbot[3225]: INFO: ::ffff:127.0.0.1:43256 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2030gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.04 seconds)2031[]2032gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2033gitlab # [ 343.543627] nixbot[3225]: INFO: ::ffff:127.0.0.1:58048 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2034gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.03 seconds)2035[]2036gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2037gitlab # [ 344.578547] nixbot[3225]: INFO: ::ffff:127.0.0.1:58062 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2038gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.03 seconds)2039[]2040gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2041gitlab # [ 345.603234] nixbot[3225]: INFO: ::ffff:127.0.0.1:58068 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2042gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2043[]2044gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2045gitlab # [ 346.625709] nixbot[3225]: INFO: ::ffff:127.0.0.1:58078 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2046gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2047[]2048gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2049gitlab # [ 347.646564] nixbot[3225]: INFO: ::ffff:127.0.0.1:58084 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2050gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2051[]2052gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2053gitlab # [ 348.670375] nixbot[3225]: INFO: ::ffff:127.0.0.1:58088 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2054gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2055[]2056gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2057gitlab # [ 349.691703] nixbot[3225]: INFO: ::ffff:127.0.0.1:58100 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2058gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2059[]2060gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2061gitlab # [ 350.714744] nixbot[3225]: INFO: ::ffff:127.0.0.1:58106 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2062gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2063[]2064gitlab # [ 351.172512] sidekiq-cluster[3241]: 2026-09-01T10:37:02.330Z pid=3241 tid=ydd INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3241, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>439914496}2065gitlab # [ 351.393872] sidekiq-cluster[3241]: W, [2026-09-01T10:37:02.551901 #3241] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.2066gitlab # [ 351.443698] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:02.601Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}2067gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2068gitlab # [ 351.758920] nixbot[3225]: INFO: ::ffff:127.0.0.1:58116 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2069gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.06 seconds)2070[]2071gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2072gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2073[]2074gitlab # [ 352.792718] nixbot[3225]: INFO: ::ffff:127.0.0.1:58130 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2075gitlab # [ 353.707910] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:04.865Z","message":"GitLab reliable fetch activated!"}2076gitlab # [ 353.711049] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:04.865Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}2077gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2078gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2079[]2080gitlab # [ 353.813441] nixbot[3225]: INFO: ::ffff:127.0.0.1:51174 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2081gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2082gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2083[]2084gitlab # [ 354.831386] nixbot[3225]: INFO: ::ffff:127.0.0.1:51182 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2085gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2086gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2087[]2088gitlab # [ 355.848533] nixbot[3225]: INFO: ::ffff:127.0.0.1:51188 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2089gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2090gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2091[]2092gitlab # [ 356.867749] nixbot[3225]: INFO: ::ffff:127.0.0.1:51192 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2093gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2094gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2095[]2096gitlab # [ 357.886442] nixbot[3225]: INFO: ::ffff:127.0.0.1:51202 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2097gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2098gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2099[]2100gitlab # [ 358.902845] nixbot[3225]: INFO: ::ffff:127.0.0.1:51206 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2101gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2102gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2103[]2104gitlab # [ 359.919362] nixbot[3225]: INFO: ::ffff:127.0.0.1:51220 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2105gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2106gitlab # [ 360.938065] nixbot[3225]: INFO: ::ffff:127.0.0.1:51230 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2107gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2108[]2109gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2110gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2111[]2112gitlab # [ 361.958674] nixbot[3225]: INFO: ::ffff:127.0.0.1:51236 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2113gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2114gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2115[]2116gitlab # [ 362.978361] nixbot[3225]: INFO: ::ffff:127.0.0.1:51248 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2117gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2118gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2119[]2120gitlab # [ 363.994273] nixbot[3225]: INFO: ::ffff:127.0.0.1:48788 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2121gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2122gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2123[]2124gitlab # [ 365.013084] nixbot[3225]: INFO: ::ffff:127.0.0.1:48804 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2125gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2126gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2127[]2128gitlab # [ 366.037358] nixbot[3225]: INFO: ::ffff:127.0.0.1:48806 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2129gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2130gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2131[]2132gitlab # [ 367.057886] nixbot[3225]: INFO: ::ffff:127.0.0.1:48812 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2133gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2134gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2135[]2136gitlab # [ 368.077785] nixbot[3225]: INFO: ::ffff:127.0.0.1:48826 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2137gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2138gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2139[]2140gitlab # [ 369.099561] nixbot[3225]: INFO: ::ffff:127.0.0.1:48832 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2141gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2142gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2143[]2144gitlab # [ 370.119245] nixbot[3225]: INFO: ::ffff:127.0.0.1:48840 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2145gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2146gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2147[]2148gitlab # [ 371.135392] nixbot[3225]: INFO: ::ffff:127.0.0.1:48846 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2149gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2150gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2151[]2152gitlab # [ 372.153234] nixbot[3225]: INFO: ::ffff:127.0.0.1:48850 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2153gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2154gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2155[]2156gitlab # [ 373.170425] nixbot[3225]: INFO: ::ffff:127.0.0.1:57422 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2157gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2158gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2159[]2160gitlab # [ 374.189392] nixbot[3225]: INFO: ::ffff:127.0.0.1:57434 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2161gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2162gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2163[]2164gitlab # [ 375.205504] nixbot[3225]: INFO: ::ffff:127.0.0.1:57438 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2165gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2166gitlab # [ 376.223205] nixbot[3225]: INFO: ::ffff:127.0.0.1:57452 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2167gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2168[]2169gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2170gitlab # [ 377.245210] nixbot[3225]: INFO: ::ffff:127.0.0.1:57468 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2171gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2172[]2173gitlab # [ 377.877891] postgres[3785]: [3785] ERROR: relation "table_without_models" does not exist2174gitlab # [ 377.879265] postgres[3785]: [3785] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')2175gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2176gitlab # [ 378.262528] nixbot[3225]: INFO: ::ffff:127.0.0.1:57474 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2177gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2178[]2179gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2180gitlab # [ 379.283329] nixbot[3225]: INFO: ::ffff:127.0.0.1:57490 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2181gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2182[]2183gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2184gitlab # [ 380.302582] nixbot[3225]: INFO: ::ffff:127.0.0.1:57492 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2185gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2186[]2187gitlab # [ 381.300055] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:32.457Z","message":"Booted Rails 7.2.3.1 application in production environment"}2188gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2189gitlab # [ 381.300982] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:32.457Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}2190gitlab # [ 381.302064] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:32.457Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}2191gitlab # [ 381.302944] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:32.457Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}2192gitlab # [ 381.304222] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:32.459Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}2193gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2194[]2195gitlab # [ 381.326589] nixbot[3225]: INFO: ::ffff:127.0.0.1:57500 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2196gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2197gitlab # [ 382.345140] nixbot[3225]: INFO: ::ffff:127.0.0.1:57504 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2198gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2199[]2200gitlab # [ 382.916883] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.074Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"dd6adffb002e22ee7ec2cac4","created_at":"2026-09-01T10:36:04.566Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"233b3894411c09b25d094bfc1d94a2f1","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.573Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-dd6adffb002e22ee7ec2cac4: start","job_status":"start","queue_duration_s":89.500976,"scheduling_latency_s":89.501024}2201gitlab # [ 382.976969] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.134Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"dd6adffb002e22ee7ec2cac4","created_at":"2026-09-01T10:36:04.566Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"233b3894411c09b25d094bfc1d94a2f1","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.573Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-dd6adffb002e22ee7ec2cac4: done: 0.060449 sec","job_status":"done","queue_duration_s":89.500976,"scheduling_latency_s":89.501024,"redis_calls":21,"redis_duration_s":0.004434,"redis_read_bytes":781,"redis_write_bytes":2288,"redis_feature_flag_calls":14,"redis_feature_flag_duration_s":0.002878,"redis_feature_flag_read_bytes":776,"redis_feature_flag_write_bytes":1607,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00114,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000416,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021638,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.060449,"completed_at":"2026-09-01T10:37:34.134Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004926,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2202gitlab # [ 382.978359] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.136Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"40639a8c762933c556c772fb","created_at":"2026-09-01T10:36:04.585Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"773cd0e64c358458082b595da046aec5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.593Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-40639a8c762933c556c772fb: start","job_status":"start","queue_duration_s":89.542896,"scheduling_latency_s":89.542923}2203gitlab # [ 382.999108] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.156Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"40639a8c762933c556c772fb","created_at":"2026-09-01T10:36:04.585Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"773cd0e64c358458082b595da046aec5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.593Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-40639a8c762933c556c772fb: done: 0.02061 sec","job_status":"done","queue_duration_s":89.542896,"scheduling_latency_s":89.542923,"redis_calls":14,"redis_duration_s":0.003096,"redis_read_bytes":13,"redis_write_bytes":1634,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000902,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":720,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001336,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000858,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000318755,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.02061,"completed_at":"2026-09-01T10:37:34.156Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000895,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2204gitlab # [ 383.046217] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.158Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"fcd1961b98fe9fab37629b27","created_at":"2026-09-01T10:36:04.608Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9fb00856fe4c5f9403844305d2bb4c2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.616Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-fcd1961b98fe9fab37629b27: start","job_status":"start","queue_duration_s":89.542402,"scheduling_latency_s":89.542425}2205gitlab # [ 383.147874] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.305Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"fcd1961b98fe9fab37629b27","created_at":"2026-09-01T10:36:04.608Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9fb00856fe4c5f9403844305d2bb4c2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.616Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-fcd1961b98fe9fab37629b27: done: 0.147165 sec","job_status":"done","queue_duration_s":89.542402,"scheduling_latency_s":89.542425,"redis_calls":15,"redis_duration_s":0.003464,"redis_read_bytes":13,"redis_write_bytes":2181,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001246,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1019,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001191,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001027,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":9,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042733,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000488889,"exclusive_lock_hold_duration_s":0.123805121,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.147165,"completed_at":"2026-09-01T10:37:34.305Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.101408,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2206gitlab # [ 383.149470] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.307Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"9abafa38af93c007af3edf91","created_at":"2026-09-01T10:36:04.627Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a66f319dde58e6f3526de4f70eb63697","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.632Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-9abafa38af93c007af3edf91: start","job_status":"start","queue_duration_s":89.674548,"scheduling_latency_s":89.674578}2207gitlab # [ 383.199111] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.356Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"9abafa38af93c007af3edf91","created_at":"2026-09-01T10:36:04.627Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a66f319dde58e6f3526de4f70eb63697","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.632Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-9abafa38af93c007af3edf91: done: 0.049428 sec","job_status":"done","queue_duration_s":89.674548,"scheduling_latency_s":89.674578,"redis_calls":20,"redis_duration_s":0.004603,"redis_read_bytes":22,"redis_write_bytes":5073,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001528,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1043,"redis_queues_calls":2,"redis_queues_duration_s":0.000379,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002042,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000654,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036268,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.049428,"completed_at":"2026-09-01T10:37:34.356Z","load_balancing_strategy":"primary","db_duration_s":0.003026,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2208gitlab # [ 383.220209] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.358Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"53ffa9e3ce6ac864361a9dcc","created_at":"2026-09-01T10:36:04.644Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7ddc3bd6866b3a26531b0396335ce1a6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.651Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-53ffa9e3ce6ac864361a9dcc: start","job_status":"start","queue_duration_s":89.707227,"scheduling_latency_s":89.707256}2209gitlab # [ 383.298043] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.371Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2210gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2211gitlab # [ 383.298929] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.377Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"53ffa9e3ce6ac864361a9dcc","created_at":"2026-09-01T10:36:04.644Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7ddc3bd6866b3a26531b0396335ce1a6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.651Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-53ffa9e3ce6ac864361a9dcc: done: 0.019549 sec","job_status":"done","queue_duration_s":89.707227,"scheduling_latency_s":89.707256,"redis_calls":11,"redis_duration_s":0.002366,"redis_read_bytes":9,"redis_write_bytes":1507,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000819,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":736,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001111,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000436,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019549,"completed_at":"2026-09-01T10:37:34.377Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000908,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2212gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.02 seconds)2213[]2214gitlab # [ 383.302051] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.379Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"6560c57c078ce5626da154bb","created_at":"2026-09-01T10:36:04.661Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9870b70df6983d5670b157e1dcdd274","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.669Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-6560c57c078ce5626da154bb: start","job_status":"start","queue_duration_s":89.709528,"scheduling_latency_s":89.709554}2215gitlab # [ 383.359770] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.420Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"6560c57c078ce5626da154bb","created_at":"2026-09-01T10:36:04.661Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9870b70df6983d5670b157e1dcdd274","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.669Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-6560c57c078ce5626da154bb: done: 0.041363 sec","job_status":"done","queue_duration_s":89.709528,"scheduling_latency_s":89.709554,"redis_calls":14,"redis_duration_s":0.004368,"redis_read_bytes":12,"redis_write_bytes":1967,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001272,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001768,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001328,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.009811582,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.041363,"completed_at":"2026-09-01T10:37:34.420Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004318,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2216gitlab # [ 383.374289] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.422Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"50d6e5983beaa02021ae095b","created_at":"2026-09-01T10:36:04.691Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21bdd6b2b872800e78df3167735267af","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.697Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-50d6e5983beaa02021ae095b: start","job_status":"start","queue_duration_s":89.724755,"scheduling_latency_s":89.724783}2217gitlab # [ 383.442652] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.476Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"50d6e5983beaa02021ae095b","created_at":"2026-09-01T10:36:04.691Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21bdd6b2b872800e78df3167735267af","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.697Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-50d6e5983beaa02021ae095b: done: 0.053634 sec","job_status":"done","queue_duration_s":89.724755,"scheduling_latency_s":89.724783,"redis_calls":13,"redis_duration_s":0.0037860000000000003,"redis_read_bytes":11,"redis_write_bytes":1921,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001219,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":752,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001279,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001288,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000835022,"exclusive_lock_hold_duration_s":0.015303342,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.053634,"completed_at":"2026-09-01T10:37:34.476Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018971,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2218gitlab # [ 383.456199] nixbot[3225]: INFO: ::ffff:127.0.0.1:38500 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2219gitlab # [ 383.457921] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.477Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"a3d4716d6ca16ac9aad1f392","created_at":"2026-09-01T10:36:04.711Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"532e8928ee8f279be33eedfe7b7a68db","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.717Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-a3d4716d6ca16ac9aad1f392: start","job_status":"start","queue_duration_s":89.760174,"scheduling_latency_s":89.760198}2220gitlab # [ 383.479186] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.503Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"a3d4716d6ca16ac9aad1f392","created_at":"2026-09-01T10:36:04.711Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"532e8928ee8f279be33eedfe7b7a68db","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.717Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-a3d4716d6ca16ac9aad1f392: done: 0.025857 sec","job_status":"done","queue_duration_s":89.760174,"scheduling_latency_s":89.760198,"redis_calls":13,"redis_duration_s":0.003167,"redis_read_bytes":11,"redis_write_bytes":1783,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001502,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1007,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001228,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000437,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016303,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025857,"completed_at":"2026-09-01T10:37:34.503Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001739,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2221gitlab # [ 383.536958] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.505Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"73bcd9b4dd4b03e0d3ed7bdf","created_at":"2026-09-01T10:36:04.735Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d20a21bef7a09217a122496c92c28389","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.742Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-73bcd9b4dd4b03e0d3ed7bdf: start","job_status":"start","queue_duration_s":89.762803,"scheduling_latency_s":89.762864}2222gitlab # [ 383.555041] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.524Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2223gitlab # [ 383.559285] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.535Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"73bcd9b4dd4b03e0d3ed7bdf","created_at":"2026-09-01T10:36:04.735Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d20a21bef7a09217a122496c92c28389","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.742Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-73bcd9b4dd4b03e0d3ed7bdf: done: 0.030121 sec","job_status":"done","queue_duration_s":89.762803,"scheduling_latency_s":89.762864,"redis_calls":11,"redis_duration_s":0.004113,"redis_read_bytes":9,"redis_write_bytes":1589,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001107,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":768,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002255,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000751,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017476,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030121,"completed_at":"2026-09-01T10:37:34.535Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001118,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2224gitlab # [ 383.614413] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.537Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"25ae3c5f7762c35f4238764f","created_at":"2026-09-01T10:36:04.763Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6d1fad37e4dfcfea1053dde3981c2aaa","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-01T10:36:04.768Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-25ae3c5f7762c35f4238764f: start","job_status":"start","queue_duration_s":89.769096,"scheduling_latency_s":89.769125}2225gitlab # [ 383.681160] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.554Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2226gitlab # [ 383.682071] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.560Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"25ae3c5f7762c35f4238764f","created_at":"2026-09-01T10:36:04.763Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6d1fad37e4dfcfea1053dde3981c2aaa","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-01T10:36:04.768Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-25ae3c5f7762c35f4238764f: done: 0.02354 sec","job_status":"done","queue_duration_s":89.769096,"scheduling_latency_s":89.769125,"redis_calls":11,"redis_duration_s":0.0025989999999999997,"redis_read_bytes":9,"redis_write_bytes":1557,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000941,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":756,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001285,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000373,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02354,"completed_at":"2026-09-01T10:37:34.560Z","load_balancing_strategy":"primary","db_duration_s":0.00098,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2227gitlab # [ 383.687357] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.562Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"93bacbc70683873dc28d5d06","created_at":"2026-09-01T10:36:04.794Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"aaf4150ae60cbfe1d9afd8f1c7623f49","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.809Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-93bacbc70683873dc28d5d06: start","job_status":"start","queue_duration_s":89.752882,"scheduling_latency_s":89.752907}2228gitlab # [ 383.706405] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.585Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"93bacbc70683873dc28d5d06","created_at":"2026-09-01T10:36:04.794Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"aaf4150ae60cbfe1d9afd8f1c7623f49","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.809Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-93bacbc70683873dc28d5d06: done: 0.023142 sec","job_status":"done","queue_duration_s":89.752882,"scheduling_latency_s":89.752907,"redis_calls":12,"redis_duration_s":0.002823,"redis_read_bytes":11,"redis_write_bytes":1594,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000978,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001411,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000434,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000449499,"duration_s":0.023142,"completed_at":"2026-09-01T10:37:34.585Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001052,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2229gitlab # [ 383.793531] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.587Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"07b273c5c8797c6c91d80934","created_at":"2026-09-01T10:36:04.823Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"632a0174ffb81db51bb556e51361bcab","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.829Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-07b273c5c8797c6c91d80934: start","job_status":"start","queue_duration_s":89.758074,"scheduling_latency_s":89.758098}2230gitlab # [ 383.794688] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.794Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"07b273c5c8797c6c91d80934","created_at":"2026-09-01T10:36:04.823Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"632a0174ffb81db51bb556e51361bcab","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.829Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-07b273c5c8797c6c91d80934: done: 0.206855 sec","job_status":"done","queue_duration_s":89.758074,"scheduling_latency_s":89.758098,"redis_calls":13,"redis_duration_s":0.003997,"redis_read_bytes":11,"redis_write_bytes":1651,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001184,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":664,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001399,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001414,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.033,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.159868,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00055454,"exclusive_lock_hold_duration_s":0.178826639,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.206855,"completed_at":"2026-09-01T10:37:34.794Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.052495,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2231gitlab # [ 383.828976] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.796Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"a36bae280b75938378f466fc","created_at":"2026-09-01T10:36:04.840Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9f30e9d52037319d55bd9869f92f557","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.846Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-a36bae280b75938378f466fc: start","job_status":"start","queue_duration_s":89.949993,"scheduling_latency_s":89.950025}2232gitlab # [ 383.847148] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.821Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"a36bae280b75938378f466fc","created_at":"2026-09-01T10:36:04.840Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9f30e9d52037319d55bd9869f92f557","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.846Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-a36bae280b75938378f466fc: done: 0.025024 sec","job_status":"done","queue_duration_s":89.949993,"scheduling_latency_s":89.950025,"redis_calls":11,"redis_duration_s":0.0026910000000000002,"redis_read_bytes":9,"redis_write_bytes":1607,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001046,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001203,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000442,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016262,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025024,"completed_at":"2026-09-01T10:37:34.821Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000921,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2233gitlab # [ 383.904036] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.823Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"71e91cbc10adf4ececa2c38d","created_at":"2026-09-01T10:36:04.860Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fd7d48af00e3c9ada0a7382f0a522b60","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-01T10:36:04.866Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-71e91cbc10adf4ececa2c38d: start","job_status":"start","queue_duration_s":89.956564,"scheduling_latency_s":89.956587}2234gitlab # [ 383.913793] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.855Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"71e91cbc10adf4ececa2c38d","created_at":"2026-09-01T10:36:04.860Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-1684658dabb34abc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fd7d48af00e3c9ada0a7382f0a522b60","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-01T10:36:04.866Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-71e91cbc10adf4ececa2c38d: done: 0.032922 sec","job_status":"done","queue_duration_s":89.956564,"scheduling_latency_s":89.956587,"redis_calls":11,"redis_duration_s":0.002778,"redis_read_bytes":9,"redis_write_bytes":1449,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000966,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":708,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001388,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000424,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021977,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032922,"completed_at":"2026-09-01T10:37:34.855Z","load_balancing_strategy":"primary","db_duration_s":0.003744,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2235gitlab # [ 384.015516] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.857Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Projects::RefreshBuildArtifactsSizeStatisticsWorker","args":[],"jid":"db53d24ac46ffdf180ed595a","created_at":"2026-09-01T10:36:05.464Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-9662ca6d37da4651","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","correlation_id":"372fb482086ae38b8ae997191c80fc3a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.475Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::RefreshBuildArtifactsSizeStatisticsWorker JID-db53d24ac46ffdf180ed595a: start","job_status":"start","queue_duration_s":89.382105,"scheduling_latency_s":89.382129}2236gitlab # [ 384.016431] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.903Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Projects::RefreshBuildArtifactsSizeStatisticsWorker","args":[],"jid":"db53d24ac46ffdf180ed595a","created_at":"2026-09-01T10:36:05.464Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-9662ca6d37da4651","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","correlation_id":"372fb482086ae38b8ae997191c80fc3a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.475Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::RefreshBuildArtifactsSizeStatisticsWorker JID-db53d24ac46ffdf180ed595a: done: 0.045682 sec","job_status":"done","queue_duration_s":89.382105,"scheduling_latency_s":89.382129,"redis_calls":19,"redis_duration_s":0.0052770000000000004,"redis_read_bytes":18,"redis_write_bytes":2650,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002439,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1330,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001303,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":519,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001535,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":801,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031043,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000477714,"duration_s":0.045682,"completed_at":"2026-09-01T10:37:34.903Z","load_balancing_strategy":"primary","db_duration_s":0.005542,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2237gitlab # [ 384.029324] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.905Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab0e7738297763432c1f110e","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab0e7738297763432c1f110e: start","job_status":"start","queue_duration_s":89.221235,"scheduling_latency_s":89.22126}2238gitlab # [ 384.045124] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.939Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab0e7738297763432c1f110e","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab0e7738297763432c1f110e: done: 0.034944 sec","job_status":"done","queue_duration_s":89.221235,"scheduling_latency_s":89.22126,"redis_calls":23,"redis_duration_s":0.0055000000000000005,"redis_read_bytes":20,"redis_write_bytes":2604,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002219,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1447,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00097,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002311,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034944,"completed_at":"2026-09-01T10:37:34.939Z","load_balancing_strategy":"primary","db_duration_s":0.002219,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2239gitlab # [ 384.095456] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.941Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0456383ff7fbf34a652dfa31","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0456383ff7fbf34a652dfa31: start","job_status":"start","queue_duration_s":89.25774,"scheduling_latency_s":89.257775}2240gitlab # [ 384.110812] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.961Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0456383ff7fbf34a652dfa31","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0456383ff7fbf34a652dfa31: done: 0.019723 sec","job_status":"done","queue_duration_s":89.25774,"scheduling_latency_s":89.257775,"redis_calls":13,"redis_duration_s":0.003185,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001065,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00212,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013223,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019723,"completed_at":"2026-09-01T10:37:34.961Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2241gitlab # [ 384.158486] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.963Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e49087831ec0390034e7a07e","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e49087831ec0390034e7a07e: start","job_status":"start","queue_duration_s":89.279156,"scheduling_latency_s":89.279181}2242gitlab # [ 384.173720] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e49087831ec0390034e7a07e","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e49087831ec0390034e7a07e: done: 0.017995 sec","job_status":"done","queue_duration_s":89.279156,"scheduling_latency_s":89.279181,"redis_calls":13,"redis_duration_s":0.002718,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000911,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001807,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017995,"completed_at":"2026-09-01T10:37:34.980Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2243gitlab # [ 384.238234] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.982Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0526f671fc2370b1260a87f","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0526f671fc2370b1260a87f: start","job_status":"start","queue_duration_s":89.298529,"scheduling_latency_s":89.298552}2244gitlab # [ 384.239224] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:34.999Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0526f671fc2370b1260a87f","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0526f671fc2370b1260a87f: done: 0.017608 sec","job_status":"done","queue_duration_s":89.298529,"scheduling_latency_s":89.298552,"redis_calls":13,"redis_duration_s":0.002569,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000864,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001705,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011995,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017608,"completed_at":"2026-09-01T10:37:34.999Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2245gitlab # [ 384.284378] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.001Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"062accaca36e720b505d6660","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-062accaca36e720b505d6660: start","job_status":"start","queue_duration_s":89.31755,"scheduling_latency_s":89.317579}2246gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2247gitlab # [ 384.348326] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.018Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"062accaca36e720b505d6660","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-062accaca36e720b505d6660: done: 0.016635 sec","job_status":"done","queue_duration_s":89.31755,"scheduling_latency_s":89.317579,"redis_calls":13,"redis_duration_s":0.002675,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000857,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001818,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011485,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016635,"completed_at":"2026-09-01T10:37:35.017Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2248gitlab # [ 384.349964] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.019Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3ef5668535786f7e64f7e92","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3ef5668535786f7e64f7e92: start","job_status":"start","queue_duration_s":89.335864,"scheduling_latency_s":89.335891}2249gitlab # [ 384.361347] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.037Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3ef5668535786f7e64f7e92","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3ef5668535786f7e64f7e92: done: 0.018101 sec","job_status":"done","queue_duration_s":89.335864,"scheduling_latency_s":89.335891,"redis_calls":13,"redis_duration_s":0.002971,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018101,"completed_at":"2026-09-01T10:37:35.037Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2250gitlab # [ 384.458902] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.039Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63e0aeebbd0faf35cd7d59ef","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63e0aeebbd0faf35cd7d59ef: start","job_status":"start","queue_duration_s":89.355428,"scheduling_latency_s":89.355483}2251gitlab # [ 384.459963] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.059Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63e0aeebbd0faf35cd7d59ef","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63e0aeebbd0faf35cd7d59ef: done: 0.020479 sec","job_status":"done","queue_duration_s":89.355428,"scheduling_latency_s":89.355483,"redis_calls":13,"redis_duration_s":0.003213,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000771,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002442,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020479,"completed_at":"2026-09-01T10:37:35.059Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2252gitlab # [ 384.472029] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.061Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e4a5e94876387d1dcca062c","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e4a5e94876387d1dcca062c: start","job_status":"start","queue_duration_s":89.37732,"scheduling_latency_s":89.377344}2253gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.19 seconds)2254[]2255gitlab # [ 384.486972] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.078Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e4a5e94876387d1dcca062c","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e4a5e94876387d1dcca062c: done: 0.017379 sec","job_status":"done","queue_duration_s":89.37732,"scheduling_latency_s":89.377344,"redis_calls":13,"redis_duration_s":0.0024850000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00088,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001605,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012221,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017379,"completed_at":"2026-09-01T10:37:35.078Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2256gitlab # [ 384.570533] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.080Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9133f08c6fbcbfb1023aa9df","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9133f08c6fbcbfb1023aa9df: start","job_status":"start","queue_duration_s":89.396178,"scheduling_latency_s":89.396203}2257gitlab # [ 384.571872] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.097Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9133f08c6fbcbfb1023aa9df","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9133f08c6fbcbfb1023aa9df: done: 0.017564 sec","job_status":"done","queue_duration_s":89.396178,"scheduling_latency_s":89.396203,"redis_calls":13,"redis_duration_s":0.002712,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000857,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001855,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011645,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017564,"completed_at":"2026-09-01T10:37:35.097Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2258gitlab # [ 384.599879] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.098Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e784a1ff363fe2bf3f5664cd","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e784a1ff363fe2bf3f5664cd: start","job_status":"start","queue_duration_s":89.415108,"scheduling_latency_s":89.415133}2259gitlab # [ 384.616289] nixbot[3225]: INFO: ::ffff:127.0.0.1:38504 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2260gitlab # [ 384.618032] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.114Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e784a1ff363fe2bf3f5664cd","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e784a1ff363fe2bf3f5664cd: done: 0.015507 sec","job_status":"done","queue_duration_s":89.415108,"scheduling_latency_s":89.415133,"redis_calls":13,"redis_duration_s":0.002558,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000763,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001795,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010628,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015507,"completed_at":"2026-09-01T10:37:35.114Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2261gitlab # [ 384.684098] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.115Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6afda614f7da9ca91869d7c8","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6afda614f7da9ca91869d7c8: start","job_status":"start","queue_duration_s":89.43195,"scheduling_latency_s":89.431972}2262gitlab # [ 384.685075] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.131Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6afda614f7da9ca91869d7c8","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6afda614f7da9ca91869d7c8: done: 0.01575 sec","job_status":"done","queue_duration_s":89.43195,"scheduling_latency_s":89.431972,"redis_calls":13,"redis_duration_s":0.00251,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00083,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00168,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010874,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01575,"completed_at":"2026-09-01T10:37:35.131Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2263gitlab # [ 384.730003] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.133Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4271dbe249796b0090f23458","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4271dbe249796b0090f23458: start","job_status":"start","queue_duration_s":89.449192,"scheduling_latency_s":89.449217}2264gitlab # [ 384.796054] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.151Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4271dbe249796b0090f23458","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4271dbe249796b0090f23458: done: 0.01808 sec","job_status":"done","queue_duration_s":89.449192,"scheduling_latency_s":89.449217,"redis_calls":13,"redis_duration_s":0.003115,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000944,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002171,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012336,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01808,"completed_at":"2026-09-01T10:37:35.151Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2265gitlab # [ 384.797054] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"410105be0f1c28c134eada89","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-410105be0f1c28c134eada89: start","job_status":"start","queue_duration_s":89.46873,"scheduling_latency_s":89.468763}2266gitlab # [ 384.808235] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"410105be0f1c28c134eada89","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-410105be0f1c28c134eada89: done: 0.015366 sec","job_status":"done","queue_duration_s":89.46873,"scheduling_latency_s":89.468763,"redis_calls":13,"redis_duration_s":0.002528,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00077,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001758,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010605,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015366,"completed_at":"2026-09-01T10:37:35.167Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2267gitlab # [ 384.906939] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.169Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3465b1769d9bbff22569b142","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3465b1769d9bbff22569b142: start","job_status":"start","queue_duration_s":89.485356,"scheduling_latency_s":89.485376}2268gitlab # [ 384.908374] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.184Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3465b1769d9bbff22569b142","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3465b1769d9bbff22569b142: done: 0.014921 sec","job_status":"done","queue_duration_s":89.485356,"scheduling_latency_s":89.485376,"redis_calls":13,"redis_duration_s":0.002381,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000775,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001606,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010471,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014921,"completed_at":"2026-09-01T10:37:35.184Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2269gitlab # [ 384.918966] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.185Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0a2c0f8663eb02ca3725f35","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0a2c0f8663eb02ca3725f35: start","job_status":"start","queue_duration_s":89.501772,"scheduling_latency_s":89.501794}2270gitlab # [ 384.934175] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.201Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0a2c0f8663eb02ca3725f35","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0a2c0f8663eb02ca3725f35: done: 0.015834 sec","job_status":"done","queue_duration_s":89.501772,"scheduling_latency_s":89.501794,"redis_calls":13,"redis_duration_s":0.00292,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000862,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002058,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015834,"completed_at":"2026-09-01T10:37:35.201Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2271gitlab # [ 385.019664] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.202Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acfd2fc8fe88b204b62ba324","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acfd2fc8fe88b204b62ba324: start","job_status":"start","queue_duration_s":89.51902,"scheduling_latency_s":89.519042}2272gitlab # [ 385.020995] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.218Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acfd2fc8fe88b204b62ba324","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acfd2fc8fe88b204b62ba324: done: 0.015731 sec","job_status":"done","queue_duration_s":89.51902,"scheduling_latency_s":89.519042,"redis_calls":13,"redis_duration_s":0.0028610000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00088,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001981,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010503,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015731,"completed_at":"2026-09-01T10:37:35.218Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2273gitlab # [ 385.073842] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.219Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cb674473ddc40d81ed3c937","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cb674473ddc40d81ed3c937: start","job_status":"start","queue_duration_s":89.536093,"scheduling_latency_s":89.536113}2274gitlab # [ 385.075046] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.235Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cb674473ddc40d81ed3c937","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cb674473ddc40d81ed3c937: done: 0.015125 sec","job_status":"done","queue_duration_s":89.536093,"scheduling_latency_s":89.536113,"redis_calls":13,"redis_duration_s":0.002989,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001112,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001877,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010272,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015125,"completed_at":"2026-09-01T10:37:35.235Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2275gitlab # [ 385.131051] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.236Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41d639aad6f2803b169c0baf","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41d639aad6f2803b169c0baf: start","job_status":"start","queue_duration_s":89.552612,"scheduling_latency_s":89.552631}2276gitlab # [ 385.131908] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.251Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41d639aad6f2803b169c0baf","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41d639aad6f2803b169c0baf: done: 0.015069 sec","job_status":"done","queue_duration_s":89.552612,"scheduling_latency_s":89.552631,"redis_calls":13,"redis_duration_s":0.002722,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000902,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00182,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010435,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015069,"completed_at":"2026-09-01T10:37:35.251Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2277gitlab # [ 385.185478] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.252Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"981d5adb3a0364a93347c25c","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-981d5adb3a0364a93347c25c: start","job_status":"start","queue_duration_s":89.568959,"scheduling_latency_s":89.568979}2278gitlab # [ 385.187837] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"981d5adb3a0364a93347c25c","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-981d5adb3a0364a93347c25c: done: 0.016826 sec","job_status":"done","queue_duration_s":89.568959,"scheduling_latency_s":89.568979,"redis_calls":13,"redis_duration_s":0.0030889999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000973,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002116,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011591,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016826,"completed_at":"2026-09-01T10:37:35.269Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2279gitlab # [ 385.241975] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64265bfccd23d2c0a582a488","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64265bfccd23d2c0a582a488: start","job_status":"start","queue_duration_s":89.58725,"scheduling_latency_s":89.587274}2280gitlab # [ 385.250535] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64265bfccd23d2c0a582a488","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64265bfccd23d2c0a582a488: done: 0.016319 sec","job_status":"done","queue_duration_s":89.58725,"scheduling_latency_s":89.587274,"redis_calls":13,"redis_duration_s":0.002707,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001777,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010905,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016319,"completed_at":"2026-09-01T10:37:35.287Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2281gitlab # [ 385.301149] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28b54d1eb9e4fd8680da6987","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28b54d1eb9e4fd8680da6987: start","job_status":"start","queue_duration_s":89.605013,"scheduling_latency_s":89.605039}2282gitlab # [ 385.354904] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.305Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28b54d1eb9e4fd8680da6987","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28b54d1eb9e4fd8680da6987: done: 0.016778 sec","job_status":"done","queue_duration_s":89.605013,"scheduling_latency_s":89.605039,"redis_calls":13,"redis_duration_s":0.002857,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000846,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002011,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011133,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016778,"completed_at":"2026-09-01T10:37:35.305Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2283gitlab # [ 385.364978] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.307Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ea7c3b87c35a74495789c2e","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ea7c3b87c35a74495789c2e: start","job_status":"start","queue_duration_s":89.623314,"scheduling_latency_s":89.623339}2284gitlab # [ 385.380493] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.323Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ea7c3b87c35a74495789c2e","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ea7c3b87c35a74495789c2e: done: 0.01626 sec","job_status":"done","queue_duration_s":89.623314,"scheduling_latency_s":89.623339,"redis_calls":13,"redis_duration_s":0.002677,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000834,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001843,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01626,"completed_at":"2026-09-01T10:37:35.323Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2285gitlab # [ 385.429864] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.324Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2c55bdd76cf1de969e3352f","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2c55bdd76cf1de969e3352f: start","job_status":"start","queue_duration_s":89.641031,"scheduling_latency_s":89.641056}2286gitlab # [ 385.445177] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.341Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2c55bdd76cf1de969e3352f","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2c55bdd76cf1de969e3352f: done: 0.016521 sec","job_status":"done","queue_duration_s":89.641031,"scheduling_latency_s":89.641056,"redis_calls":13,"redis_duration_s":0.0033049999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00148,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001825,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016521,"completed_at":"2026-09-01T10:37:35.341Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2287gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2288gitlab # [ 385.496991] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.342Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93b5beb0861b890999aff743","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93b5beb0861b890999aff743: start","job_status":"start","queue_duration_s":89.658901,"scheduling_latency_s":89.658924}2289gitlab # [ 385.513277] gitaly[916]: time="2026-09-01T10:37:36.558Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:37:46.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:36.552" grpc.time_ms=5.5 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2290gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.04 seconds)2291[]2292gitlab # [ 385.571853] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.357Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93b5beb0861b890999aff743","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93b5beb0861b890999aff743: done: 0.015112 sec","job_status":"done","queue_duration_s":89.658901,"scheduling_latency_s":89.658924,"redis_calls":13,"redis_duration_s":0.002611,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000791,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00182,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010207,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015112,"completed_at":"2026-09-01T10:37:35.357Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2293gitlab # [ 385.577586] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.359Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e52be2bb6b130e03e07a57d","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e52be2bb6b130e03e07a57d: start","job_status":"start","queue_duration_s":89.675241,"scheduling_latency_s":89.67526}2294gitlab # [ 385.593460] gitaly[916]: time="2026-09-01T10:37:36.568Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-01T10:37:46.012" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:37:36.563" grpc.time_ms=5.328 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2295gitlab # [ 385.610372] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.375Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e52be2bb6b130e03e07a57d","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e52be2bb6b130e03e07a57d: done: 0.016088 sec","job_status":"done","queue_duration_s":89.675241,"scheduling_latency_s":89.67526,"redis_calls":13,"redis_duration_s":0.002684,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000795,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001889,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011091,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016088,"completed_at":"2026-09-01T10:37:35.375Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2296gitlab # [ 385.660238] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7615445981f9f652a65ac6df","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7615445981f9f652a65ac6df: start","job_status":"start","queue_duration_s":89.692878,"scheduling_latency_s":89.692902}2297gitlab # [ 385.676439] nixbot[3225]: INFO: ::ffff:127.0.0.1:38518 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2298gitlab # [ 385.678373] gitaly[916]: time="2026-09-01T10:37:36.613Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-01T10:37:46.050" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:36.601" grpc.time_ms=11.781 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2299gitlab # [ 385.695877] gitaly[916]: time="2026-09-01T10:37:36.645Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3933 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:37:36.644" grpc.time_ms=0.359 pid=916 span.kind=server system=grpc2300gitlab # [ 385.707964] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.393Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7615445981f9f652a65ac6df","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7615445981f9f652a65ac6df: done: 0.016981 sec","job_status":"done","queue_duration_s":89.692878,"scheduling_latency_s":89.692902,"redis_calls":13,"redis_duration_s":0.003097,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000999,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016981,"completed_at":"2026-09-01T10:37:35.393Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2301gitlab # [ 385.756568] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.395Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1dffea939f638e0e02a5534","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1dffea939f638e0e02a5534: start","job_status":"start","queue_duration_s":89.711477,"scheduling_latency_s":89.711502}2302gitlab # [ 385.814883] gitaly[916]: time="2026-09-01T10:37:36.669Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3935 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:37:36.668" grpc.time_ms=0.565 pid=916 span.kind=server system=grpc2303gitlab # [ 385.816253] gitaly[916]: time="2026-09-01T10:37:36.676Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K diskcache=3530e1ef-ecef-46a4-bca9-9e7c378d60b2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-01T10:37:46.073" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:37:36.624" pid=916 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root2304gitlab # [ 385.817160] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.411Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1dffea939f638e0e02a5534","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1dffea939f638e0e02a5534: done: 0.016313 sec","job_status":"done","queue_duration_s":89.711477,"scheduling_latency_s":89.711502,"redis_calls":13,"redis_duration_s":0.002592,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000783,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001809,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016313,"completed_at":"2026-09-01T10:37:35.411Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2305gitlab # [ 385.845443] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.412Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66d1724ea78b52374b63d4ad","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66d1724ea78b52374b63d4ad: start","job_status":"start","queue_duration_s":89.729135,"scheduling_latency_s":89.729158}2306gitlab # [ 385.861180] gitaly[916]: time="2026-09-01T10:37:36.676Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=47 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4649 command.oublock=8 command.real_time_ms=46 command.spawn_token_fork_ms=1 command.system_time_ms=31 command.user_time_ms=16 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-01T10:37:46.073" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:37:36.624" grpc.time_ms=51.841 pid=916 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root2307gitlab # [ 385.928247] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66d1724ea78b52374b63d4ad","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66d1724ea78b52374b63d4ad: done: 0.016686 sec","job_status":"done","queue_duration_s":89.729135,"scheduling_latency_s":89.729158,"redis_calls":13,"redis_duration_s":0.0028539999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001001,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001853,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016686,"completed_at":"2026-09-01T10:37:35.429Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2308gitlab # [ 385.930090] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"837947874b896c2a5e7fc23e","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-837947874b896c2a5e7fc23e: start","job_status":"start","queue_duration_s":89.747234,"scheduling_latency_s":89.747258}2309gitlab # [ 385.942220] gitaly[916]: time="2026-09-01T10:37:36.695Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:37:46.144" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:37:36.695" grpc.time_ms=0.238 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2310gitlab # [ 385.956756] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.447Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"837947874b896c2a5e7fc23e","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-837947874b896c2a5e7fc23e: done: 0.016474 sec","job_status":"done","queue_duration_s":89.747234,"scheduling_latency_s":89.747258,"redis_calls":13,"redis_duration_s":0.00268,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001804,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01096,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016474,"completed_at":"2026-09-01T10:37:35.447Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2311gitlab # [ 386.052560] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.448Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6870e110ece43da3fb5727db","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6870e110ece43da3fb5727db: start","job_status":"start","queue_duration_s":89.76509,"scheduling_latency_s":89.765113}2312gitlab # [ 386.053855] gitaly[916]: time="2026-09-01T10:37:36.707Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=944 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-01T10:37:46.150" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:36.700" grpc.time_ms=6.785 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2313gitlab # [ 386.055238] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.464Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6870e110ece43da3fb5727db","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6870e110ece43da3fb5727db: done: 0.01602 sec","job_status":"done","queue_duration_s":89.76509,"scheduling_latency_s":89.765113,"redis_calls":13,"redis_duration_s":0.00245,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000754,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001696,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01602,"completed_at":"2026-09-01T10:37:35.464Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2314gitlab # [ 386.105960] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.466Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45a8e605ab8b632c35313bd6","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45a8e605ab8b632c35313bd6: start","job_status":"start","queue_duration_s":89.782482,"scheduling_latency_s":89.782505}2315gitlab # [ 386.125093] gitaly[916]: time="2026-09-01T10:37:36.795Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=343 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-01T10:38:06.086" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:36.787" grpc.time_ms=7.681 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2316gitlab # [ 386.186073] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.480Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45a8e605ab8b632c35313bd6","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45a8e605ab8b632c35313bd6: done: 0.014671 sec","job_status":"done","queue_duration_s":89.782482,"scheduling_latency_s":89.782505,"redis_calls":13,"redis_duration_s":0.00241,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000703,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001707,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009944,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014671,"completed_at":"2026-09-01T10:37:35.480Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2317gitlab # [ 386.198391] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.482Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84b797704baf0a99f26a893a","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84b797704baf0a99f26a893a: start","job_status":"start","queue_duration_s":89.798292,"scheduling_latency_s":89.798311}2318gitlab # [ 386.214086] gitaly[916]: time="2026-09-01T10:37:36.824Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-01T10:37:46.272" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:37:36.823" grpc.time_ms=0.625 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2319gitlab # [ 386.227316] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.497Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84b797704baf0a99f26a893a","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84b797704baf0a99f26a893a: done: 0.01523 sec","job_status":"done","queue_duration_s":89.798292,"scheduling_latency_s":89.798311,"redis_calls":13,"redis_duration_s":0.002463,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001706,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010247,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01523,"completed_at":"2026-09-01T10:37:35.497Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2320gitlab # [ 386.276095] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.498Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e9a6549e3d96031c4114602","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e9a6549e3d96031c4114602: start","job_status":"start","queue_duration_s":89.815014,"scheduling_latency_s":89.815038}2321gitlab # [ 386.291483] gitaly[916]: time="2026-09-01T10:37:36.835Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=342 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-01T10:37:46.276" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-01T10:37:36.827" grpc.time_ms=6.924 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2322gitlab # [ 386.308147] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.515Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e9a6549e3d96031c4114602","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e9a6549e3d96031c4114602: done: 0.016241 sec","job_status":"done","queue_duration_s":89.815014,"scheduling_latency_s":89.815038,"redis_calls":13,"redis_duration_s":0.002683,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000884,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001799,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010834,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016241,"completed_at":"2026-09-01T10:37:35.515Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2323gitlab # [ 386.361911] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.516Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d616b70884c0449e72883fd","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d616b70884c0449e72883fd: start","job_status":"start","queue_duration_s":89.832987,"scheduling_latency_s":89.833017}2324gitlab # [ 386.377089] gitaly[916]: time="2026-09-01T10:37:36.855Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:37:46.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:36.849" grpc.time_ms=4.871 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2325gitlab # [ 386.392628] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.532Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d616b70884c0449e72883fd","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d616b70884c0449e72883fd: done: 0.016052 sec","job_status":"done","queue_duration_s":89.832987,"scheduling_latency_s":89.833017,"redis_calls":13,"redis_duration_s":0.002584,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000837,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001747,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010625,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016052,"completed_at":"2026-09-01T10:37:35.532Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2326gitlab # [ 386.441794] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.534Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0bf477bd4d42e27df133e379","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0bf477bd4d42e27df133e379: start","job_status":"start","queue_duration_s":89.850411,"scheduling_latency_s":89.850433}2327gitlab # [ 386.460976] gitaly[916]: time="2026-09-01T10:37:36.868Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:37:46.020" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:36.861" grpc.time_ms=6.948 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2328gitlab # [ 386.479867] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.553Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0bf477bd4d42e27df133e379","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0bf477bd4d42e27df133e379: done: 0.018943 sec","job_status":"done","queue_duration_s":89.850411,"scheduling_latency_s":89.850433,"redis_calls":13,"redis_duration_s":0.0030789999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000911,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002168,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012491,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018943,"completed_at":"2026-09-01T10:37:35.553Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2329gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2330gitlab # [ 386.533491] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.555Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81b57142eae5d00336ac28ea","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81b57142eae5d00336ac28ea: start","job_status":"start","queue_duration_s":89.871194,"scheduling_latency_s":89.871224}2331gitlab # [ 386.548400] gitaly[916]: time="2026-09-01T10:37:36.892Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=961 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-01T10:38:06.181" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:36.881" grpc.time_ms=11.165 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2332gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.03 seconds)2333[]2334gitlab # [ 386.568038] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.573Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81b57142eae5d00336ac28ea","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81b57142eae5d00336ac28ea: done: 0.01844 sec","job_status":"done","queue_duration_s":89.871194,"scheduling_latency_s":89.871224,"redis_calls":13,"redis_duration_s":0.002578,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000799,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001779,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013037,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01844,"completed_at":"2026-09-01T10:37:35.573Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2335gitlab # [ 386.656336] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.574Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2aad6e3c28ba880b5bf5e6db","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2aad6e3c28ba880b5bf5e6db: start","job_status":"start","queue_duration_s":89.891079,"scheduling_latency_s":89.891109}2336gitlab # [ 386.657362] gitaly[916]: time="2026-09-01T10:37:37.329Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-01T10:37:47.007" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:37.328" grpc.time_ms=1.276 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2337gitlab # [ 386.658190] nixbot[3225]: INFO: ::ffff:127.0.0.1:38530 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2338gitlab # [ 386.658844] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.592Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2aad6e3c28ba880b5bf5e6db","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2aad6e3c28ba880b5bf5e6db: done: 0.017558 sec","job_status":"done","queue_duration_s":89.891079,"scheduling_latency_s":89.891109,"redis_calls":13,"redis_duration_s":0.0023610000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000799,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001562,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012303,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017558,"completed_at":"2026-09-01T10:37:35.592Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2339gitlab # [ 386.704945] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ed1dfcd8475f62daa526b7c","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ed1dfcd8475f62daa526b7c: start","job_status":"start","queue_duration_s":89.910027,"scheduling_latency_s":89.910051}2340gitlab # [ 386.792543] gitaly[916]: time="2026-09-01T10:37:37.390Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:37:47.065" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:37.385" grpc.time_ms=4.858 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2341gitlab # [ 386.793928] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.610Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ed1dfcd8475f62daa526b7c","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ed1dfcd8475f62daa526b7c: done: 0.016997 sec","job_status":"done","queue_duration_s":89.910027,"scheduling_latency_s":89.910051,"redis_calls":13,"redis_duration_s":0.002719,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000989,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00173,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016997,"completed_at":"2026-09-01T10:37:35.610Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2342gitlab # [ 386.810505] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.612Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29aa739f5dab686f69f0bf0e","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29aa739f5dab686f69f0bf0e: start","job_status":"start","queue_duration_s":89.928594,"scheduling_latency_s":89.928619}2343gitlab # [ 386.828232] gitaly[916]: time="2026-09-01T10:37:37.401Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-01T10:37:47.075" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:37:37.396" grpc.time_ms=4.652 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2344gitlab # [ 386.848843] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.629Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29aa739f5dab686f69f0bf0e","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29aa739f5dab686f69f0bf0e: done: 0.017207 sec","job_status":"done","queue_duration_s":89.928594,"scheduling_latency_s":89.928619,"redis_calls":13,"redis_duration_s":0.002899,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000882,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01175,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017207,"completed_at":"2026-09-01T10:37:35.629Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2345gitlab # [ 386.900947] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.631Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b93f225ca51e9174269f3c3a","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b93f225ca51e9174269f3c3a: start","job_status":"start","queue_duration_s":89.947197,"scheduling_latency_s":89.94722}2346gitlab # [ 386.916117] gitaly[916]: time="2026-09-01T10:37:37.425Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-01T10:37:47.097" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:37.418" grpc.time_ms=7.307 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2347gitlab # [ 386.933865] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.646Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b93f225ca51e9174269f3c3a","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b93f225ca51e9174269f3c3a: done: 0.015487 sec","job_status":"done","queue_duration_s":89.947197,"scheduling_latency_s":89.94722,"redis_calls":13,"redis_duration_s":0.002805,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000997,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001808,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015487,"completed_at":"2026-09-01T10:37:35.646Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2348gitlab # [ 387.033112] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.647Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e8a2b2596b69488915e5f602","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e8a2b2596b69488915e5f602: start","job_status":"start","queue_duration_s":89.963975,"scheduling_latency_s":89.963993}2349gitlab # [ 387.034235] gitaly[916]: time="2026-09-01T10:37:37.439Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=654 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-01T10:38:07.030" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:37.430" grpc.time_ms=8.405 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2350gitlab # [ 387.035234] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.663Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e8a2b2596b69488915e5f602","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e8a2b2596b69488915e5f602: done: 0.015633 sec","job_status":"done","queue_duration_s":89.963975,"scheduling_latency_s":89.963993,"redis_calls":13,"redis_duration_s":0.003031,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001173,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001858,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010501,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015633,"completed_at":"2026-09-01T10:37:35.663Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2351gitlab # [ 387.063089] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.664Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61f570ef6a4f5d5ace7bd7e2","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61f570ef6a4f5d5ace7bd7e2: start","job_status":"start","queue_duration_s":89.980827,"scheduling_latency_s":89.980847}2352gitlab # [ 387.078521] gitaly[916]: time="2026-09-01T10:37:37.453Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=344 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-01T10:37:47.129" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-01T10:37:37.450" grpc.time_ms=3.329 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2353gitlab # [ 387.094422] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.679Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61f570ef6a4f5d5ace7bd7e2","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61f570ef6a4f5d5ace7bd7e2: done: 0.01473 sec","job_status":"done","queue_duration_s":89.980827,"scheduling_latency_s":89.980847,"redis_calls":13,"redis_duration_s":0.002387,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000762,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001625,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01473,"completed_at":"2026-09-01T10:37:35.679Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2354gitlab # [ 387.152196] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0fc83542ee93e91890f7066","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0fc83542ee93e91890f7066: start","job_status":"start","queue_duration_s":89.996882,"scheduling_latency_s":89.996902}2355gitlab # [ 387.157526] gitaly[916]: time="2026-09-01T10:37:37.467Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:37:47.141" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:37.462" grpc.time_ms=4.936 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2356gitlab # [ 387.176143] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0fc83542ee93e91890f7066","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0fc83542ee93e91890f7066: done: 0.014528 sec","job_status":"done","queue_duration_s":89.996882,"scheduling_latency_s":89.996902,"redis_calls":13,"redis_duration_s":0.002456,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000735,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001721,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014528,"completed_at":"2026-09-01T10:37:35.695Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2357gitlab # [ 387.240433] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.696Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa6b7711c03d32fb7785cd4b","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa6b7711c03d32fb7785cd4b: start","job_status":"start","queue_duration_s":90.012861,"scheduling_latency_s":90.012883}2358gitlab # [ 387.241504] gitaly[916]: time="2026-09-01T10:37:37.535Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-01T10:37:47.212" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:37.533" grpc.time_ms=1.974 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2359gitlab # [ 387.257999] gitaly[916]: time="2026-09-01T10:37:37.875Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:37:37.873" grpc.time_ms=1.7 pid=916 span.kind=server system=grpc2360gitlab # [ 387.271505] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.713Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa6b7711c03d32fb7785cd4b","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa6b7711c03d32fb7785cd4b: done: 0.017246 sec","job_status":"done","queue_duration_s":90.012861,"scheduling_latency_s":90.012883,"redis_calls":13,"redis_duration_s":0.003023,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001066,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001957,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011692,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017246,"completed_at":"2026-09-01T10:37:35.713Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2361gitlab # [ 387.327524] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.715Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11333a5ad2193e077e4d4483","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11333a5ad2193e077e4d4483: start","job_status":"start","queue_duration_s":90.032024,"scheduling_latency_s":90.032057}2362gitlab # [ 387.385160] gitaly[916]: time="2026-09-01T10:37:37.895Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:37:37.895" grpc.time_ms=0.474 pid=916 span.kind=server system=grpc2363gitlab # [ 387.386149] gitaly[916]: time="2026-09-01T10:37:37.917Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:37:37.917" grpc.time_ms=0.471 pid=916 span.kind=server system=grpc2364gitlab # [ 387.387105] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.735Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11333a5ad2193e077e4d4483","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11333a5ad2193e077e4d4483: done: 0.020014 sec","job_status":"done","queue_duration_s":90.032024,"scheduling_latency_s":90.032057,"redis_calls":13,"redis_duration_s":0.003404,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002313,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013183,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020014,"completed_at":"2026-09-01T10:37:35.735Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2365gitlab # [ 387.416633] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.737Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4bafaa60d63854ca758badd","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4bafaa60d63854ca758badd: start","job_status":"start","queue_duration_s":90.054018,"scheduling_latency_s":90.05405}2366gitlab # [ 387.483042] gitaly[916]: time="2026-09-01T10:37:37.923Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS diskcache=0d76d093-2e3c-4484-85a2-25712dc2a173 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T10:38:07.153" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:37.853" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2367gitlab # [ 387.484148] gitaly[916]: time="2026-09-01T10:37:37.924Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=70 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7435 command.oublock=0 command.real_time_ms=65 command.spawn_token_fork_ms=1 command.system_time_ms=42 command.user_time_ms=28 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T10:38:07.153" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:37.853" grpc.time_ms=70.391 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2368gitlab # [ 387.485246] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4bafaa60d63854ca758badd","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4bafaa60d63854ca758badd: done: 0.019558 sec","job_status":"done","queue_duration_s":90.054018,"scheduling_latency_s":90.05405,"redis_calls":13,"redis_duration_s":0.003042,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002121,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013052,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019558,"completed_at":"2026-09-01T10:37:35.757Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2369gitlab # [ 387.507865] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61f7c0f69564a7375ebd747a","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61f7c0f69564a7375ebd747a: start","job_status":"start","queue_duration_s":90.075126,"scheduling_latency_s":90.075152}2370gitlab # [ 387.523275] gitaly[916]: time="2026-09-01T10:37:38.038Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-01T10:38:08.035" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:38.036" grpc.time_ms=1.252 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2371gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2372gitlab # [ 387.539145] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.779Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61f7c0f69564a7375ebd747a","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61f7c0f69564a7375ebd747a: done: 0.021017 sec","job_status":"done","queue_duration_s":90.075126,"scheduling_latency_s":90.075152,"redis_calls":13,"redis_duration_s":0.003648,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001418,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00223,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014637,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021017,"completed_at":"2026-09-01T10:37:35.779Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2373gitlab # [ 387.588901] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.781Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"899efc79348620c0b536e65a","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-899efc79348620c0b536e65a: start","job_status":"start","queue_duration_s":90.097794,"scheduling_latency_s":90.097819}2374gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.03 seconds)2375[]2376gitlab # [ 387.604356] gitaly[916]: time="2026-09-01T10:37:38.073Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:08.070" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:38.072" grpc.time_ms=0.862 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2377gitlab # [ 387.619359] gitaly[916]: time="2026-09-01T10:37:38.234Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-01T16:37:38.228" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:37:38.229" grpc.time_ms=3.98 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2378gitlab # [ 387.632490] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.803Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"899efc79348620c0b536e65a","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-899efc79348620c0b536e65a: done: 0.022367 sec","job_status":"done","queue_duration_s":90.097794,"scheduling_latency_s":90.097819,"redis_calls":13,"redis_duration_s":0.003994,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001809,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002185,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015069,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022367,"completed_at":"2026-09-01T10:37:35.803Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2379gitlab # [ 387.720452] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.805Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fbc37039868b828d2842d7e3","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fbc37039868b828d2842d7e3: start","job_status":"start","queue_duration_s":90.121891,"scheduling_latency_s":90.121914}2380gitlab # [ 387.721498] gitaly[916]: time="2026-09-01T10:37:38.249Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=652 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-01T10:38:08.242" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:38.243" grpc.time_ms=6.201 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2381gitlab # [ 387.722719] gitaly[916]: time="2026-09-01T10:37:38.317Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K diskcache=ab84ec8c-e280-415c-ad84-53a13a405c02 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-01T16:37:38.298" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:38.299" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2382gitlab # [ 387.723854] nixbot[3225]: INFO: ::ffff:127.0.0.1:38540 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2383gitlab # [ 387.724739] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.821Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fbc37039868b828d2842d7e3","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fbc37039868b828d2842d7e3: done: 0.015984 sec","job_status":"done","queue_duration_s":90.121891,"scheduling_latency_s":90.121914,"redis_calls":13,"redis_duration_s":0.002397,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000767,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00163,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010615,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015984,"completed_at":"2026-09-01T10:37:35.821Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2384gitlab # [ 387.774428] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.823Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55d65146e570adb61ff37d11","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55d65146e570adb61ff37d11: start","job_status":"start","queue_duration_s":90.13941,"scheduling_latency_s":90.139496}2385gitlab # [ 387.832898] gitaly[916]: time="2026-09-01T10:37:38.317Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1285 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-01T16:37:38.298" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:38.299" grpc.time_ms=17.512 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2386gitlab # [ 387.833910] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55d65146e570adb61ff37d11","created_at":"2026-09-01T10:36:05.565Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-37575965c5ac4cdc","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"426804debe64b6bc55fb52398ed405c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.683Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55d65146e570adb61ff37d11: done: 0.020935 sec","job_status":"done","queue_duration_s":90.13941,"scheduling_latency_s":90.139496,"redis_calls":13,"redis_duration_s":0.003175,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000856,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002319,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020935,"completed_at":"2026-09-01T10:37:35.844Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2387gitlab # [ 387.855982] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.845Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"358c946bf3be3d0253fae448","created_at":"2026-09-01T10:36:12.483Z","trace_propagation_headers":{"sentry-trace":"182a8ff4c8eb405dbf340b98b0d462c6-7e1bda5478734338","baggage":"sentry-trace_id=182a8ff4c8eb405dbf340b98b0d462c6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8MVASVNWQYQKMYHM5YM7S","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:12.527Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-358c946bf3be3d0253fae448: start","job_status":"start","queue_duration_s":83.318218,"scheduling_latency_s":83.318244}2388gitlab # [ 387.874814] gitaly[916]: time="2026-09-01T10:37:38.391Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-01T10:38:08.385" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:38.386" grpc.time_ms=4.828 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2389gitlab # [ 387.943262] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.952Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"358c946bf3be3d0253fae448","created_at":"2026-09-01T10:36:12.483Z","trace_propagation_headers":{"sentry-trace":"182a8ff4c8eb405dbf340b98b0d462c6-7e1bda5478734338","baggage":"sentry-trace_id=182a8ff4c8eb405dbf340b98b0d462c6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8MVASVNWQYQKMYHM5YM7S","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:12.527Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-358c946bf3be3d0253fae448: done: 0.107019 sec","job_status":"done","queue_duration_s":83.318218,"scheduling_latency_s":83.318244,"redis_calls":13,"redis_duration_s":0.003341,"redis_read_bytes":138,"redis_write_bytes":1593,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000716,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":428,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001434,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001191,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.070113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.072217559,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.107019,"completed_at":"2026-09-01T10:37:35.952Z","load_balancing_strategy":"primary","db_duration_s":0.029139,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2390gitlab # [ 387.955758] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:35.954Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"26584dcf699446167cb440e9","created_at":"2026-09-01T10:36:12.542Z","trace_propagation_headers":{"sentry-trace":"182a8ff4c8eb405dbf340b98b0d462c6-7e1bda5478734338","baggage":"sentry-trace_id=182a8ff4c8eb405dbf340b98b0d462c6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8MVASVNWQYQKMYHM5YM7S","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:12.553Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-26584dcf699446167cb440e9: start","job_status":"start","queue_duration_s":83.401051,"scheduling_latency_s":83.401074}2391gitlab # [ 387.974494] gitaly[916]: time="2026-09-01T10:37:38.429Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:08.428" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:38.428" grpc.time_ms=0.576 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2392gitlab # [ 387.990287] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:36.037Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"26584dcf699446167cb440e9","created_at":"2026-09-01T10:36:12.542Z","trace_propagation_headers":{"sentry-trace":"182a8ff4c8eb405dbf340b98b0d462c6-7e1bda5478734338","baggage":"sentry-trace_id=182a8ff4c8eb405dbf340b98b0d462c6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8MVASVNWQYQKMYHM5YM7S","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:12.553Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-26584dcf699446167cb440e9: done: 0.083501 sec","job_status":"done","queue_duration_s":83.401051,"scheduling_latency_s":83.401074,"redis_calls":14,"redis_duration_s":0.003183,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000698,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001487,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.06231,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.061369252,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.083501,"completed_at":"2026-09-01T10:37:36.037Z","load_balancing_strategy":"primary","db_duration_s":0.014027,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2393gitlab # [ 388.047275] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:36.039Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"ff4c71fdfed47678b56384fc","created_at":"2026-09-01T10:36:13.592Z","trace_propagation_headers":{"sentry-trace":"182a8ff4c8eb405dbf340b98b0d462c6-7e1bda5478734338","baggage":"sentry-trace_id=182a8ff4c8eb405dbf340b98b0d462c6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8MVASVNWQYQKMYHM5YM7S","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:13.604Z","job_size_bytes":5,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-ff4c71fdfed47678b56384fc: start","job_status":"start","queue_duration_s":82.434819,"scheduling_latency_s":82.434843}2394gitlab # [ 388.069482] gitaly[916]: time="2026-09-01T10:37:38.438Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-01T10:37:48.112" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:38.432" grpc.time_ms=6.088 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2395gitlab # [ 388.086430] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:36.194Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"ff4c71fdfed47678b56384fc","created_at":"2026-09-01T10:36:13.592Z","trace_propagation_headers":{"sentry-trace":"182a8ff4c8eb405dbf340b98b0d462c6-7e1bda5478734338","baggage":"sentry-trace_id=182a8ff4c8eb405dbf340b98b0d462c6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8MVASVNWQYQKMYHM5YM7S","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:13.604Z","job_size_bytes":5,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-ff4c71fdfed47678b56384fc: done: 0.155066 sec","job_status":"done","queue_duration_s":82.434819,"scheduling_latency_s":82.434843,"redis_calls":12,"redis_duration_s":0.002812,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000664,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001718,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00043,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.125449,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000453409,"duration_s":0.155066,"completed_at":"2026-09-01T10:37:36.194Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020042,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2396gitlab # [ 388.144144] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:36.196Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"499199fff589a9714cfb3c74","created_at":"2026-09-01T10:36:13.645Z","trace_propagation_headers":{"sentry-trace":"182a8ff4c8eb405dbf340b98b0d462c6-7e1bda5478734338","baggage":"sentry-trace_id=182a8ff4c8eb405dbf340b98b0d462c6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8MVASVNWQYQKMYHM5YM7S","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:13.654Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-499199fff589a9714cfb3c74: start","job_status":"start","queue_duration_s":82.541646,"scheduling_latency_s":82.541677}2397gitlab # [ 388.163449] gitaly[916]: time="2026-09-01T10:37:38.539Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-01T10:38:08.538" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:38.538" grpc.time_ms=0.85 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2398gitlab # [ 388.179482] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:36.464Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"499199fff589a9714cfb3c74","created_at":"2026-09-01T10:36:13.645Z","trace_propagation_headers":{"sentry-trace":"182a8ff4c8eb405dbf340b98b0d462c6-7e1bda5478734338","baggage":"sentry-trace_id=182a8ff4c8eb405dbf340b98b0d462c6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8MVASVNWQYQKMYHM5YM7S","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:13.654Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-499199fff589a9714cfb3c74: done: 0.268183 sec","job_status":"done","queue_duration_s":82.541646,"scheduling_latency_s":82.541677,"redis_calls":11,"redis_duration_s":0.002568,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00073,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00138,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000458,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.169,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.169,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.20438,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000372673,"duration_s":0.268183,"completed_at":"2026-09-01T10:37:36.464Z","load_balancing_strategy":"primary","db_duration_s":0.063581,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2399gitlab # [ 388.234135] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:36.466Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"c0c5253cba0f950233395a33","created_at":"2026-09-01T10:36:13.660Z","trace_propagation_headers":{"sentry-trace":"182a8ff4c8eb405dbf340b98b0d462c6-7e1bda5478734338","baggage":"sentry-trace_id=182a8ff4c8eb405dbf340b98b0d462c6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8MVASVNWQYQKMYHM5YM7S","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:13.668Z","job_size_bytes":89,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-c0c5253cba0f950233395a33: start","job_status":"start","queue_duration_s":82.797291,"scheduling_latency_s":82.797315}2400gitlab # [ 388.300037] gitaly[916]: time="2026-09-01T10:37:38.548Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=226 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-01T10:38:08.543" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:38.543" grpc.time_ms=3.307 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2401gitlab # [ 388.301148] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:36.500Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"c0c5253cba0f950233395a33","created_at":"2026-09-01T10:36:13.660Z","trace_propagation_headers":{"sentry-trace":"182a8ff4c8eb405dbf340b98b0d462c6-7e1bda5478734338","baggage":"sentry-trace_id=182a8ff4c8eb405dbf340b98b0d462c6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8MVASVNWQYQKMYHM5YM7S","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:13.668Z","job_size_bytes":89,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-c0c5253cba0f950233395a33: done: 0.034597 sec","job_status":"done","queue_duration_s":82.797291,"scheduling_latency_s":82.797315,"redis_calls":11,"redis_duration_s":0.00228,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.0006,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001264,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000416,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023051,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000359263,"duration_s":0.034597,"completed_at":"2026-09-01T10:37:36.500Z","load_balancing_strategy":"primary","db_duration_s":0.004447,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2402gitlab # [ 388.330316] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:36.502Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 ae86dcce1d51f83b4a05ba73d940f28faca8fab9 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"e40f4c7704f077cf38f6d5c7","created_at":"2026-09-01T10:36:17.364Z","trace_propagation_headers":{"sentry-trace":"9747fac109a543f6bf57052f34116f48-2075c29e5a764307","baggage":"sentry-trace_id=9747fac109a543f6bf57052f34116f48,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8MZ3DZ3D1SWJBZTBSPN0K","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:17.377Z","job_size_bytes":185,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-e40f4c7704f077cf38f6d5c7: start","job_status":"start","queue_duration_s":79.125353,"scheduling_latency_s":79.125379}2403gitlab # [ 388.389282] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:36.519Z","message":"changes: \"0000000000000000000000000000000000000000 ae86dcce1d51f83b4a05ba73d940f28faca8fab9 refs/heads/master\\n\""}2404gitlab # [ 388.390660] gitaly[916]: time="2026-09-01T10:37:38.701Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:37:48.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:38.696" grpc.time_ms=5.066 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2405gitlab # [ 388.391721] nixbot[3225]: INFO: ::ffff:127.0.0.1:38546 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2406gitlab # [ 388.392709] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:37.354Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 ae86dcce1d51f83b4a05ba73d940f28faca8fab9 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"e40f4c7704f077cf38f6d5c7","created_at":"2026-09-01T10:36:17.364Z","trace_propagation_headers":{"sentry-trace":"9747fac109a543f6bf57052f34116f48-2075c29e5a764307","baggage":"sentry-trace_id=9747fac109a543f6bf57052f34116f48,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8MZ3DZ3D1SWJBZTBSPN0K","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:17.377Z","job_size_bytes":185,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-e40f4c7704f077cf38f6d5c7: done: 0.851871 sec","job_status":"done","queue_duration_s":79.125353,"scheduling_latency_s":79.125379,"gitaly_calls":13,"gitaly_duration_s":0.168541,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.042985,"redis_read_bytes":2107,"redis_write_bytes":21752,"redis_cache_calls":15,"redis_cache_duration_s":0.003983,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.006489,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.00148,"redis_queues_read_bytes":14,"redis_queues_write_bytes":7840,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002791,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.018719,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.009523,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.041,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.229,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.249,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.041,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.589216,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000926934,"exclusive_lock_hold_duration_s":0.027736485999999998,"duration_s":0.851871,"completed_at":"2026-09-01T10:37:37.354Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.072549,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2407gitlab # [ 388.478903] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:37.356Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","ae86dcce1d51f83b4a05ba73d940f28faca8fab9 d996dce18654c3c835478cedbe8d54682367b5ea refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"73df064755f075f2c2df0ba3","created_at":"2026-09-01T10:36:26.520Z","trace_propagation_headers":{"sentry-trace":"58684ea44cea4ffda557b00e4cb59895-e1c06d71e0114300","baggage":"sentry-trace_id=58684ea44cea4ffda557b00e4cb59895,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8NA0VZXEZDXQDHQ9N8F0K","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:26.543Z","job_size_bytes":185,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-73df064755f075f2c2df0ba3: start","job_status":"start","queue_duration_s":70.813135,"scheduling_latency_s":70.813159}2408gitlab # [ 388.479906] gitaly[916]: time="2026-09-01T10:37:39.243Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=966 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-01T10:38:09.233" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:39.233" grpc.time_ms=9.45 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2409gitlab # [ 388.480773] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:37.368Z","message":"changes: \"ae86dcce1d51f83b4a05ba73d940f28faca8fab9 d996dce18654c3c835478cedbe8d54682367b5ea refs/heads/master\\n\""}2410gitlab # [ 388.481594] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:37.405Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"5a29bdc91cb17ac9ff2ae221","created_at":"2026-09-01T10:37:37.404Z","trace_propagation_headers":{"sentry-trace":"58684ea44cea4ffda557b00e4cb59895-ea037d359ace4e06","baggage":"sentry-trace_id=58684ea44cea4ffda557b00e4cb59895,sentry-release=Unknown"},"correlation_id":"01M1E8NA0VZXEZDXQDHQ9N8F0K","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"b9dfa4d4e017dd92844225e8","job_size_bytes":46,"pid":3241,"job_status":"deduplicated","message":"ProjectCacheWorker JID-5a29bdc91cb17ac9ff2ae221: deduplicated: until executing","deduplication.type":"until executing"}2411gitlab # [ 388.495181] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:37.550Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","ae86dcce1d51f83b4a05ba73d940f28faca8fab9 d996dce18654c3c835478cedbe8d54682367b5ea refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"73df064755f075f2c2df0ba3","created_at":"2026-09-01T10:36:26.520Z","trace_propagation_headers":{"sentry-trace":"58684ea44cea4ffda557b00e4cb59895-e1c06d71e0114300","baggage":"sentry-trace_id=58684ea44cea4ffda557b00e4cb59895,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8NA0VZXEZDXQDHQ9N8F0K","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:26.543Z","job_size_bytes":185,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-73df064755f075f2c2df0ba3: done: 0.194595 sec","job_status":"done","queue_duration_s":70.813135,"scheduling_latency_s":70.813159,"gitaly_calls":7,"gitaly_duration_s":0.056088,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.018585,"redis_read_bytes":501,"redis_write_bytes":11942,"redis_cache_calls":9,"redis_cache_duration_s":0.002061,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.000763,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5719,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.00234,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.006841,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.00658,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.017,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.017,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.120507,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000738641,"exclusive_lock_hold_duration_s":0.014697119,"duration_s":0.194595,"completed_at":"2026-09-01T10:37:37.550Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019053,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2412gitlab # [ 388.572094] gitaly[916]: time="2026-09-01T10:37:39.258Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8MZ3DZ3D1SWJBZTBSPN0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-01T10:38:09.246" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:39.246" grpc.time_ms=11.213 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2413gitlab # [ 388.573401] gitaly[916]: time="2026-09-01T10:37:39.585Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K diskcache=167c6d1e-c0c2-4d8a-9efa-82ae8f8ffc8e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-01T16:37:39.572" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:39.574" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2414gitlab # [ 388.636692] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1372415gitlab # [ 388.637856] gitlab-workhorse[826]: time="2026-09-01T10:37:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672416gitlab # [ 388.638745] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 672417gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2418gitlab # [ 388.639704] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:37.552Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"760e1ff7525f746b08b98c11","created_at":"2026-09-01T10:36:27.280Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-34eda1acf7774465","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:27.286Z","job_size_bytes":65,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-760e1ff7525f746b08b98c11: start","job_status":"start","queue_duration_s":70.265418,"scheduling_latency_s":70.265441}2419gitlab # [ 388.640770] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:37.631Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"760e1ff7525f746b08b98c11","created_at":"2026-09-01T10:36:27.280Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-34eda1acf7774465","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:27.286Z","job_size_bytes":65,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-760e1ff7525f746b08b98c11: done: 0.078713 sec","job_status":"done","queue_duration_s":70.265418,"scheduling_latency_s":70.265441,"redis_calls":16,"redis_duration_s":0.004046,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002107,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001478,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000461,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052803,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000388038,"duration_s":0.078713,"completed_at":"2026-09-01T10:37:37.630Z","load_balancing_strategy":"primary","db_duration_s":0.018057,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2420gitlab # [ 388.668247] gitaly[916]: time="2026-09-01T10:37:39.586Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=314 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-01T16:37:39.572" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:39.574" grpc.time_ms=11.412 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2421gitlab # [ 388.683814] nixbot[3225]: {"ts": "2026-09-01T10:37:39.761Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}2422gitlab # [ 388.686921] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:37.632Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"4f3464856e985fab1b9a9dda","created_at":"2026-09-01T10:36:27.287Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-34eda1acf7774465","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:27.292Z","job_size_bytes":65,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-4f3464856e985fab1b9a9dda: start","job_status":"start","queue_duration_s":70.340652,"scheduling_latency_s":70.340676}2423gitlab # [ 388.711111] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:37.725Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"4f3464856e985fab1b9a9dda","created_at":"2026-09-01T10:36:27.287Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-34eda1acf7774465","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:27.292Z","job_size_bytes":65,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-4f3464856e985fab1b9a9dda: done: 0.093132 sec","job_status":"done","queue_duration_s":70.340652,"scheduling_latency_s":70.340676,"redis_calls":21,"redis_duration_s":0.004853,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.001904,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0015,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001449,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.067199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000396698,"duration_s":0.093132,"completed_at":"2026-09-01T10:37:37.725Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017893,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2424gitlab # [ 388.771079] gitaly[916]: time="2026-09-01T10:37:39.626Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:09.624" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:39.625" grpc.time_ms=1.622 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2425gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.18 seconds)2426[]2427gitlab # [ 388.786489] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:37.728Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"535d50ae40eb4880e94f6bfa","created_at":"2026-09-01T10:36:27.391Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-34eda1acf7774465","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:27.397Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-535d50ae40eb4880e94f6bfa: start","job_status":"start","queue_duration_s":70.331711,"scheduling_latency_s":70.331743}2428gitlab # [ 388.808233] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:38.083Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"535d50ae40eb4880e94f6bfa","created_at":"2026-09-01T10:36:27.391Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-34eda1acf7774465","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:27.397Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3241,"job_status":"rescheduled","message":"PipelineProcessWorker JID-535d50ae40eb4880e94f6bfa: rescheduled"}2429gitlab # [ 388.869748] gitaly[916]: time="2026-09-01T10:37:39.634Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-01T10:37:49.008" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:39.628" grpc.time_ms=5.278 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2430gitlab # [ 388.871252] nixbot[3225]: INFO: ::ffff:127.0.0.1:38552 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2431gitlab # [ 388.872266] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:38.091Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"535d50ae40eb4880e94f6bfa","created_at":"2026-09-01T10:36:27.391Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-34eda1acf7774465","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:27.397Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-535d50ae40eb4880e94f6bfa: done: 0.362778 sec","job_status":"done","queue_duration_s":70.331711,"scheduling_latency_s":70.331743,"gitaly_calls":3,"gitaly_duration_s":0.081825,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.027344,"redis_read_bytes":463,"redis_write_bytes":29294,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001395,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001176,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.009779,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.004725,"redis_queues_read_bytes":40,"redis_queues_write_bytes":15581,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.005569,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000479,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000592,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002234,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.046,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.017,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.019,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.023,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.213416,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.304577817,"duration_s":0.362778,"completed_at":"2026-09-01T10:37:38.091Z","load_balancing_strategy":"primary","db_duration_s":0.054961,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2432gitlab # [ 388.911125] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:38.093Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0c3551bbe1cc02f7294d579e","created_at":"2026-09-01T10:37:34.327Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94494700f78a42c3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a66f319dde58e6f3526de4f70eb63697","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:34.349Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0c3551bbe1cc02f7294d579e: start","job_status":"start","queue_duration_s":3.743284,"scheduling_latency_s":3.743307}2433gitlab # [ 388.926913] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:38.109Z","message":"No existing merge request to be cleaned up."}2434gitlab # [ 388.928821] gitaly[916]: time="2026-09-01T10:37:39.673Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8QHTGH01TE6N2BQZ7ZD7T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:37:39.664" grpc.time_ms=8.411 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc2435gitlab # [ 388.945579] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:38.120Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0c3551bbe1cc02f7294d579e","created_at":"2026-09-01T10:37:34.327Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94494700f78a42c3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a66f319dde58e6f3526de4f70eb63697","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:34.349Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0c3551bbe1cc02f7294d579e: done: 0.026796 sec","job_status":"done","queue_duration_s":3.743284,"scheduling_latency_s":3.743307,"redis_calls":12,"redis_duration_s":0.0028250000000000003,"redis_read_bytes":10,"redis_write_bytes":1323,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000479,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000955,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001391,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017137,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026796,"completed_at":"2026-09-01T10:37:38.119Z","load_balancing_strategy":"primary","db_duration_s":0.00344,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2436gitlab # [ 388.996466] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:38.121Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6fb98342b8914443b601cd3e","created_at":"2026-09-01T10:37:34.327Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94494700f78a42c3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a66f319dde58e6f3526de4f70eb63697","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:34.349Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6fb98342b8914443b601cd3e: start","job_status":"start","queue_duration_s":3.771675,"scheduling_latency_s":3.771701}2437gitlab # [ 389.011954] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:38.133Z","message":"No existing merge request to be cleaned up."}2438gitlab # [ 389.013842] gitaly[916]: time="2026-09-01T10:37:39.681Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-01T10:38:09.680" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:39.680" grpc.time_ms=0.68 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2439gitlab # [ 389.030456] nixbot[3225]: {"ts": "2026-09-01T10:37:40.113Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ae86dcce1d51f83b4a05ba73d940f28faca8fab9 \"HTTP/1.1 201 Created\""}2440gitlab # [ 389.033669] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:38.140Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6fb98342b8914443b601cd3e","created_at":"2026-09-01T10:37:34.327Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94494700f78a42c3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a66f319dde58e6f3526de4f70eb63697","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:34.349Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6fb98342b8914443b601cd3e: done: 0.019339 sec","job_status":"done","queue_duration_s":3.771675,"scheduling_latency_s":3.771701,"redis_calls":10,"redis_duration_s":0.00236,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001006,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001354,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012488,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019339,"completed_at":"2026-09-01T10:37:38.140Z","load_balancing_strategy":"primary","db_duration_s":0.002159,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2441gitlab # [ 389.098108] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:38.142Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5baae9b0725c559e3c4e763f","created_at":"2026-09-01T10:37:34.327Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94494700f78a42c3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a66f319dde58e6f3526de4f70eb63697","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:34.349Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5baae9b0725c559e3c4e763f: start","job_status":"start","queue_duration_s":3.792543,"scheduling_latency_s":3.792567}2442gitlab # [ 389.099874] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:38.153Z","message":"No existing merge request to be cleaned up."}2443gitlab # [ 389.100909] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:40 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ae86dcce1d51f83b4a05ba73d940f28faca8fab9 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 3392444gitlab # [ 389.101780] gitaly[916]: time="2026-09-01T10:37:39.691Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=324 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-01T10:38:09.686" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:39.686" grpc.time_ms=4.346 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2445gitlab # [ 389.159033] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:38.164Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5baae9b0725c559e3c4e763f","created_at":"2026-09-01T10:37:34.327Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94494700f78a42c3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a66f319dde58e6f3526de4f70eb63697","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:34.349Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5baae9b0725c559e3c4e763f: done: 0.021601 sec","job_status":"done","queue_duration_s":3.792543,"scheduling_latency_s":3.792567,"redis_calls":10,"redis_duration_s":0.002688,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001076,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001612,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014259,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021601,"completed_at":"2026-09-01T10:37:38.164Z","load_balancing_strategy":"primary","db_duration_s":0.002268,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2446gitlab # [ 389.167968] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:38.166Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b2745eccf3ee636940db2367","created_at":"2026-09-01T10:37:34.327Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94494700f78a42c3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a66f319dde58e6f3526de4f70eb63697","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:34.349Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b2745eccf3ee636940db2367: start","job_status":"start","queue_duration_s":3.816033,"scheduling_latency_s":3.816063}2447gitlab # [ 389.183716] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:38.179Z","message":"No existing merge request to be cleaned up."}2448gitlab # [ 389.185958] nixbot[3225]: {"ts": "2026-09-01T10:37:40.286Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ae86dcce1d51f83b4a05ba73d940f28faca8fab9 \"HTTP/1.1 201 Created\""}2449gitlab # [ 389.189464] gitaly[916]: time="2026-09-01T10:37:39.725Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:37:49.028" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:39.718" grpc.time_ms=6.946 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2450gitlab # [ 389.205706] gitaly[916]: time="2026-09-01T10:37:39.742Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QHYV22471DB2RFKJVKBJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:37:39.731" pid=916 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2451gitlab # [ 389.218178] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:40 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ae86dcce1d51f83b4a05ba73d940f28faca8fab9 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1662452gitlab # [ 389.221400] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:38.186Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b2745eccf3ee636940db2367","created_at":"2026-09-01T10:37:34.327Z","trace_propagation_headers":{"sentry-trace":"48bae9b25ada4d45bf804674daaf1c7d-94494700f78a42c3","baggage":"sentry-trace_id=48bae9b25ada4d45bf804674daaf1c7d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a66f319dde58e6f3526de4f70eb63697","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:34.349Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b2745eccf3ee636940db2367: done: 0.020939 sec","job_status":"done","queue_duration_s":3.816033,"scheduling_latency_s":3.816063,"redis_calls":10,"redis_duration_s":0.002213,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000868,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001345,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013807,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020939,"completed_at":"2026-09-01T10:37:38.186Z","load_balancing_strategy":"primary","db_duration_s":0.002123,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2453gitlab # [ 389.323585] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:38.188Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"b9dfa4d4e017dd92844225e8","created_at":"2026-09-01T10:37:36.571Z","trace_propagation_headers":{"sentry-trace":"9747fac109a543f6bf57052f34116f48-437d994ed4db4e18","baggage":"sentry-trace_id=9747fac109a543f6bf57052f34116f48,sentry-release=Unknown"},"correlation_id":"01M1E8MZ3DZ3D1SWJBZTBSPN0K","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:36.576Z","job_size_bytes":46,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-b9dfa4d4e017dd92844225e8: start","job_status":"start","queue_duration_s":1.612166,"scheduling_latency_s":1.61219}2454gitlab # [ 389.324819] gitaly[916]: time="2026-09-01T10:37:39.742Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QHYV22471DB2RFKJVKBJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:37:39.731" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc2455gitlab # [ 389.328214] nixbot[3225]: INFO: ::ffff:127.0.0.1:38554 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2456gitlab # [ 389.329101] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:38.275Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"b9dfa4d4e017dd92844225e8","created_at":"2026-09-01T10:37:36.571Z","trace_propagation_headers":{"sentry-trace":"9747fac109a543f6bf57052f34116f48-437d994ed4db4e18","baggage":"sentry-trace_id=9747fac109a543f6bf57052f34116f48,sentry-release=Unknown"},"correlation_id":"01M1E8MZ3DZ3D1SWJBZTBSPN0K","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:36.576Z","job_size_bytes":46,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-b9dfa4d4e017dd92844225e8: done: 0.086797 sec","job_status":"done","queue_duration_s":1.612166,"scheduling_latency_s":1.61219,"gitaly_calls":2,"gitaly_duration_s":0.016136,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.007216,"redis_read_bytes":69,"redis_write_bytes":4887,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001381,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000534,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2115,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001697,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.002846,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000758,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053685,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004224,"duration_s":0.086797,"completed_at":"2026-09-01T10:37:38.275Z","load_balancing_strategy":"primary","db_duration_s":0.010541,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2457gitlab # [ 389.371689] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:38.276Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"a1ab1bdca2175aa313efff3c","created_at":"2026-09-01T10:37:36.590Z","trace_propagation_headers":{"sentry-trace":"9747fac109a543f6bf57052f34116f48-437d994ed4db4e18","baggage":"sentry-trace_id=9747fac109a543f6bf57052f34116f48,sentry-release=Unknown"},"correlation_id":"01M1E8MZ3DZ3D1SWJBZTBSPN0K","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:36.596Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-a1ab1bdca2175aa313efff3c: start","job_status":"start","queue_duration_s":1.680703,"scheduling_latency_s":1.680725}2458gitlab # [ 389.443213] gitaly[916]: time="2026-09-01T10:37:39.742Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QHYV22471DB2RFKJVKBJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:37:39.731" grpc.time_ms=10.628 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc2459gitlab # [ 389.444710] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:40 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 922460gitlab # [ 389.446915] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:38.354Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"a1ab1bdca2175aa313efff3c","created_at":"2026-09-01T10:37:36.590Z","trace_propagation_headers":{"sentry-trace":"9747fac109a543f6bf57052f34116f48-437d994ed4db4e18","baggage":"sentry-trace_id=9747fac109a543f6bf57052f34116f48,sentry-release=Unknown"},"correlation_id":"01M1E8MZ3DZ3D1SWJBZTBSPN0K","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:36.596Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-a1ab1bdca2175aa313efff3c: done: 0.077858 sec","job_status":"done","queue_duration_s":1.680703,"scheduling_latency_s":1.680725,"gitaly_calls":1,"gitaly_duration_s":0.020579,"redis_calls":15,"redis_duration_s":0.003477,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000444,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001303,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000603,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001127,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.011,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.02,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037724,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.054849149,"duration_s":0.077858,"completed_at":"2026-09-01T10:37:38.354Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01459,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2461gitlab # [ 389.472895] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:38.356Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"2cf0fbc4ca3c6ef40f2951a9","created_at":"2026-09-01T10:37:36.690Z","trace_propagation_headers":{"sentry-trace":"9747fac109a543f6bf57052f34116f48-437d994ed4db4e18","baggage":"sentry-trace_id=9747fac109a543f6bf57052f34116f48,sentry-release=Unknown"},"correlation_id":"01M1E8MZ3DZ3D1SWJBZTBSPN0K","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:36.692Z","job_size_bytes":23,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-2cf0fbc4ca3c6ef40f2951a9: start","job_status":"start","queue_duration_s":1.664233,"scheduling_latency_s":1.664268}2462gitlab # [ 389.492817] gitaly[916]: time="2026-09-01T10:37:39.829Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QJ1W7J9DVA9Z4626ETC3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:09.627" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:39.828" grpc.time_ms=1.302 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2463gitlab # [ 389.556989] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:38.399Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"2cf0fbc4ca3c6ef40f2951a9","created_at":"2026-09-01T10:37:36.690Z","trace_propagation_headers":{"sentry-trace":"9747fac109a543f6bf57052f34116f48-437d994ed4db4e18","baggage":"sentry-trace_id=9747fac109a543f6bf57052f34116f48,sentry-release=Unknown"},"correlation_id":"01M1E8MZ3DZ3D1SWJBZTBSPN0K","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:36.692Z","job_size_bytes":23,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-2cf0fbc4ca3c6ef40f2951a9: done: 0.043269 sec","job_status":"done","queue_duration_s":1.664233,"scheduling_latency_s":1.664268,"gitaly_calls":1,"gitaly_duration_s":0.007013,"redis_calls":16,"redis_duration_s":0.004344,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000286,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001194,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.002058,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000806,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026355,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043269,"completed_at":"2026-09-01T10:37:38.399Z","load_balancing_strategy":"primary","db_duration_s":0.003012,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2464gitlab # [ 389.565046] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:38.401Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"5714c0616515e14b129c299b","created_at":"2026-09-01T10:37:37.165Z","trace_propagation_headers":{"sentry-trace":"9747fac109a543f6bf57052f34116f48-437d994ed4db4e18","baggage":"sentry-trace_id=9747fac109a543f6bf57052f34116f48,sentry-release=Unknown"},"correlation_id":"01M1E8MZ3DZ3D1SWJBZTBSPN0K","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c0b1b757d63f961136a5838b5325c1431899b1feb2123ce586cb28977f69189e","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.170Z","job_size_bytes":467,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-5714c0616515e14b129c299b: start","job_status":"start","queue_duration_s":1.231637,"scheduling_latency_s":1.231663}2465gitlab # [ 389.585500] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:40 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ae86dcce1d51f83b4a05ba73d940f28faca8fab9 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 4062466gitlab # [ 389.588974] gitlab-workhorse[826]: time="2026-09-01T10:37:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672467gitlab # [ 389.590889] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:40 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1632468gitlab # [ 389.593884] gitaly[916]: time="2026-09-01T10:37:40.044Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8QJ1W7J9DVA9Z4626ETC3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:37:50.024" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:40.034" grpc.time_ms=9.683 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2469gitlab # [ 389.611081] nixbot[3225]: {"ts": "2026-09-01T10:37:40.698Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ae86dcce1d51f83b4a05ba73d940f28faca8fab9 \"HTTP/1.1 201 Created\""}2470gitlab # [ 389.667177] sidekiq-cluster[3241]: {"severity":"WARN","time":"2026-09-01T10:37:39.287Z","class":"CreatePipelineWorker","correlation_id":"01M1E8MZ3DZ3D1SWJBZTBSPN0K","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"ae86dcce1d51f83b4a05ba73d940f28faca8fab9","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"ae86dcce1d51f83b4a05ba73d940f28faca8fab9","ref":"refs/heads/master"},"retry":0}2471gitlab # [ 389.668133] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:39.295Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"5714c0616515e14b129c299b","created_at":"2026-09-01T10:37:37.165Z","trace_propagation_headers":{"sentry-trace":"9747fac109a543f6bf57052f34116f48-437d994ed4db4e18","baggage":"sentry-trace_id=9747fac109a543f6bf57052f34116f48,sentry-release=Unknown"},"correlation_id":"01M1E8MZ3DZ3D1SWJBZTBSPN0K","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c0b1b757d63f961136a5838b5325c1431899b1feb2123ce586cb28977f69189e","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.170Z","job_size_bytes":467,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-5714c0616515e14b129c299b: done: 0.893586 sec","job_status":"done","queue_duration_s":1.231637,"scheduling_latency_s":1.231663,"gitaly_calls":7,"gitaly_duration_s":0.057415,"redis_calls":51,"redis_duration_s":0.010394,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.000876,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.004438,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001646,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.00244,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000994,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.778335,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000445867,"duration_s":0.893586,"completed_at":"2026-09-01T10:37:39.295Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.040854,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2472gitlab # [ 389.689991] gitaly[916]: time="2026-09-01T10:37:40.183Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QJCPEPEK0VEWYGQXDCG1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:10.082" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:40.182" grpc.time_ms=0.742 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2473gitlab # [ 389.701476] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:40 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 712474gitlab # [ 389.703645] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:39.297Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"c4cda05d-1f40-4dce-b437-ecef208745b5\"}"],"class":"WebHookWorker","jid":"6aacd8b8ef73e9e7cb709123","created_at":"2026-09-01T10:37:37.299Z","trace_propagation_headers":{"sentry-trace":"9747fac109a543f6bf57052f34116f48-437d994ed4db4e18","baggage":"sentry-trace_id=9747fac109a543f6bf57052f34116f48,sentry-release=Unknown"},"correlation_id":"01M1E8MZ3DZ3D1SWJBZTBSPN0K","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f45dd869d589f9f9eff7624fc90988470b15dcc8a5cef1e30bcd1a0fdd337998","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.304Z","job_size_bytes":1751,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-6aacd8b8ef73e9e7cb709123: start","job_status":"start","queue_duration_s":1.992658,"scheduling_latency_s":1.992712}2475gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2476gitlab # [ 389.781157] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:39.544Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"c4cda05d-1f40-4dce-b437-ecef208745b5\"}"],"class":"WebHookWorker","jid":"6aacd8b8ef73e9e7cb709123","created_at":"2026-09-01T10:37:37.299Z","trace_propagation_headers":{"sentry-trace":"9747fac109a543f6bf57052f34116f48-437d994ed4db4e18","baggage":"sentry-trace_id=9747fac109a543f6bf57052f34116f48,sentry-release=Unknown"},"correlation_id":"01M1E8MZ3DZ3D1SWJBZTBSPN0K","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f45dd869d589f9f9eff7624fc90988470b15dcc8a5cef1e30bcd1a0fdd337998","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.304Z","job_size_bytes":1751,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-6aacd8b8ef73e9e7cb709123: done: 0.247337 sec","job_status":"done","queue_duration_s":1.992658,"scheduling_latency_s":1.992712,"redis_calls":19,"redis_duration_s":0.005221,"redis_read_bytes":32,"redis_write_bytes":5367,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001194,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000412,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002265,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00135,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.04727164999997058,"cpu_s":0.180125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000598121,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.247337,"completed_at":"2026-09-01T10:37:39.544Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003706,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2477gitlab # [ 389.792976] gitaly[916]: time="2026-09-01T10:37:40.223Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8QJCPEPEK0VEWYGQXDCG1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:37:50.206" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:40.217" grpc.time_ms=5.578 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2478gitlab # [ 389.809362] gitlab-workhorse[826]: time="2026-09-01T10:37:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672479gitlab # [ 389.811207] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:40 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 812480gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 0.03 seconds)2481[]2482gitlab # [ 389.814087] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:39.547Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"cfd75e567a372f2dc5aa206a","created_at":"2026-09-01T10:37:37.411Z","trace_propagation_headers":{"sentry-trace":"58684ea44cea4ffda557b00e4cb59895-ea037d359ace4e06","baggage":"sentry-trace_id=58684ea44cea4ffda557b00e4cb59895,sentry-release=Unknown"},"correlation_id":"01M1E8NA0VZXEZDXQDHQ9N8F0K","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"a1ab1bdca2175aa313efff3c","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.413Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-cfd75e567a372f2dc5aa206a: start","job_status":"start","queue_duration_s":2.134125,"scheduling_latency_s":2.13416}2483gitlab # [ 389.836574] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:39.600Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"cfd75e567a372f2dc5aa206a","created_at":"2026-09-01T10:37:37.411Z","trace_propagation_headers":{"sentry-trace":"58684ea44cea4ffda557b00e4cb59895-ea037d359ace4e06","baggage":"sentry-trace_id=58684ea44cea4ffda557b00e4cb59895,sentry-release=Unknown"},"correlation_id":"01M1E8NA0VZXEZDXQDHQ9N8F0K","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"a1ab1bdca2175aa313efff3c","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.413Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-cfd75e567a372f2dc5aa206a: done: 0.052723 sec","job_status":"done","queue_duration_s":2.134125,"scheduling_latency_s":2.13416,"gitaly_calls":1,"gitaly_duration_s":0.015597,"redis_calls":12,"redis_duration_s":0.004661,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001252,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001985,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001424,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.029506823,"duration_s":0.052723,"completed_at":"2026-09-01T10:37:39.600Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003917,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2484gitlab # [ 389.907972] gitaly[916]: time="2026-09-01T10:37:40.314Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=964 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=4 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-01T10:38:10.303" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:40.304" grpc.time_ms=10.311 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2485gitlab # [ 389.913689] nixbot[3225]: INFO: ::ffff:127.0.0.1:38566 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2486gitlab # [ 389.915620] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:39.602Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"5a86db48f7e6ac4c666f4a61","created_at":"2026-09-01T10:37:37.506Z","trace_propagation_headers":{"sentry-trace":"58684ea44cea4ffda557b00e4cb59895-ea037d359ace4e06","baggage":"sentry-trace_id=58684ea44cea4ffda557b00e4cb59895,sentry-release=Unknown"},"correlation_id":"01M1E8NA0VZXEZDXQDHQ9N8F0K","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8fae7dc79894906b369b2d305a777ba475f950d535c99ed9b7b56d9c3463c9f0","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.507Z","job_size_bytes":467,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-5a86db48f7e6ac4c666f4a61: start","job_status":"start","queue_duration_s":2.094914,"scheduling_latency_s":2.094937}2487gitlab # [ 389.977498] sidekiq-cluster[3241]: {"severity":"WARN","time":"2026-09-01T10:37:40.374Z","class":"CreatePipelineWorker","correlation_id":"01M1E8NA0VZXEZDXQDHQ9N8F0K","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"ae86dcce1d51f83b4a05ba73d940f28faca8fab9","after":"d996dce18654c3c835478cedbe8d54682367b5ea","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"d996dce18654c3c835478cedbe8d54682367b5ea","ref":"refs/heads/master"},"retry":0}2488gitlab # [ 389.979049] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:41 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 792489gitlab # [ 389.980175] gitlab-workhorse[826]: time="2026-09-01T10:37:41Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2162490gitlab # [ 389.981064] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:41 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 772491gitlab # [ 389.982046] gitaly[916]: time="2026-09-01T10:37:40.344Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=20 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NA0VZXEZDXQDHQ9N8F0K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-01T10:38:10.317" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:40.318" grpc.time_ms=26.705 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2492gitlab # [ 389.983073] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:40.381Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"5a86db48f7e6ac4c666f4a61","created_at":"2026-09-01T10:37:37.506Z","trace_propagation_headers":{"sentry-trace":"58684ea44cea4ffda557b00e4cb59895-ea037d359ace4e06","baggage":"sentry-trace_id=58684ea44cea4ffda557b00e4cb59895,sentry-release=Unknown"},"correlation_id":"01M1E8NA0VZXEZDXQDHQ9N8F0K","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8fae7dc79894906b369b2d305a777ba475f950d535c99ed9b7b56d9c3463c9f0","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.507Z","job_size_bytes":467,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-5a86db48f7e6ac4c666f4a61: done: 0.778427 sec","job_status":"done","queue_duration_s":2.094914,"scheduling_latency_s":2.094937,"gitaly_calls":7,"gitaly_duration_s":0.075497,"redis_calls":33,"redis_duration_s":0.006923,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.001841,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001239,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002872,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000971,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.035,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.637172,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.778427,"completed_at":"2026-09-01T10:37:40.380Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.037828,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2493gitlab # [ 390.036331] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:40.382Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"15c78857-b69a-442a-986b-5d2bf41ff3b9\"}"],"class":"WebHookWorker","jid":"db128f8054e797b151adc32f","created_at":"2026-09-01T10:37:37.519Z","trace_propagation_headers":{"sentry-trace":"58684ea44cea4ffda557b00e4cb59895-ea037d359ace4e06","baggage":"sentry-trace_id=58684ea44cea4ffda557b00e4cb59895,sentry-release=Unknown"},"correlation_id":"01M1E8NA0VZXEZDXQDHQ9N8F0K","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b2fea468d5cebf94cbc9e94417e47b48172eeefda7e0f6a08745b81a6e00100b","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.521Z","job_size_bytes":1749,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-db128f8054e797b151adc32f: start","job_status":"start","queue_duration_s":2.861385,"scheduling_latency_s":2.861408}2494gitlab # [ 390.059286] gitaly[916]: time="2026-09-01T10:37:40.480Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QJJ2KRSP536K05VXJ5BE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:10.279" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:40.479" grpc.time_ms=0.564 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2495gitlab # [ 390.076808] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:40.445Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"15c78857-b69a-442a-986b-5d2bf41ff3b9\"}"],"class":"WebHookWorker","jid":"db128f8054e797b151adc32f","created_at":"2026-09-01T10:37:37.519Z","trace_propagation_headers":{"sentry-trace":"58684ea44cea4ffda557b00e4cb59895-ea037d359ace4e06","baggage":"sentry-trace_id=58684ea44cea4ffda557b00e4cb59895,sentry-release=Unknown"},"correlation_id":"01M1E8NA0VZXEZDXQDHQ9N8F0K","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b2fea468d5cebf94cbc9e94417e47b48172eeefda7e0f6a08745b81a6e00100b","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.521Z","job_size_bytes":1749,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-db128f8054e797b151adc32f: done: 0.062277 sec","job_status":"done","queue_duration_s":2.861385,"scheduling_latency_s":2.861408,"redis_calls":14,"redis_duration_s":0.0041990000000000005,"redis_read_bytes":26,"redis_write_bytes":4684,"redis_queues_calls":2,"redis_queues_duration_s":0.000439,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002713,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001047,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.007160966000014923,"cpu_s":0.031583,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.062277,"completed_at":"2026-09-01T10:37:40.445Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00191,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2496gitlab # [ 390.155479] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:40.448Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"82d6aabf3272224c0db63a0e","created_at":"2026-09-01T10:37:37.840Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.844Z","job_size_bytes":23,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-82d6aabf3272224c0db63a0e: start","job_status":"start","queue_duration_s":2.603499,"scheduling_latency_s":2.603548}2497gitlab # [ 390.162485] gitaly[916]: time="2026-09-01T10:37:40.551Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=29 command.spawn_token_fork_ms=20 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8QJQCGMH3YC2DE2G4MNAG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:37:40.519" grpc.time_ms=32.042 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc2498gitlab # [ 390.227219] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:40.553Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"82d6aabf3272224c0db63a0e","created_at":"2026-09-01T10:37:37.840Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.844Z","job_size_bytes":23,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-82d6aabf3272224c0db63a0e: done: 0.105856 sec","job_status":"done","queue_duration_s":2.603499,"scheduling_latency_s":2.603548,"redis_calls":10,"redis_duration_s":0.005863,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000499,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0041,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001264,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.070423,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000480229,"duration_s":0.105856,"completed_at":"2026-09-01T10:37:40.553Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015098,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2499gitlab # [ 390.237576] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:40.556Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"f2a817a7e395b3d5050383c7","created_at":"2026-09-01T10:37:37.929Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.935Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-f2a817a7e395b3d5050383c7: start","job_status":"start","queue_duration_s":2.621625,"scheduling_latency_s":2.621651}2500gitlab # [ 390.328534] gitaly[916]: time="2026-09-01T10:37:40.602Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=4 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8QJJ2KRSP536K05VXJ5BE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:37:50.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:40.594" grpc.time_ms=7.54 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2501gitlab # [ 390.329988] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:40.602Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"f2a817a7e395b3d5050383c7","created_at":"2026-09-01T10:37:37.929Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.935Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-f2a817a7e395b3d5050383c7: done: 0.045625 sec","job_status":"done","queue_duration_s":2.621625,"scheduling_latency_s":2.621651,"redis_calls":10,"redis_duration_s":0.002983,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000565,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00139,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001028,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027079,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000538337,"duration_s":0.045625,"completed_at":"2026-09-01T10:37:40.602Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004557,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2502gitlab # [ 390.359363] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:40.604Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"8ae188648ac56be739926435","created_at":"2026-09-01T10:37:37.945Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.950Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-8ae188648ac56be739926435: start","job_status":"start","queue_duration_s":2.65459,"scheduling_latency_s":2.654616}2503gitlab # [ 390.380073] gitaly[916]: time="2026-09-01T10:37:40.717Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QJTA6WPFWSKEM9GZPQVQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:37:40.704" pid=916 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2504gitlab # [ 390.393124] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:40.653Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"8ae188648ac56be739926435","created_at":"2026-09-01T10:37:37.945Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.950Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-8ae188648ac56be739926435: done: 0.049367 sec","job_status":"done","queue_duration_s":2.65459,"scheduling_latency_s":2.654616,"redis_calls":10,"redis_duration_s":0.00978,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005274,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003673,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000833,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027073,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001719492,"duration_s":0.049367,"completed_at":"2026-09-01T10:37:40.653Z","load_balancing_strategy":"primary","db_duration_s":0.005226,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2505gitlab # [ 390.493336] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:40.655Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"d5bd102fe1e8cb381032dff4","created_at":"2026-09-01T10:37:37.962Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.967Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-d5bd102fe1e8cb381032dff4: start","job_status":"start","queue_duration_s":2.688265,"scheduling_latency_s":2.688289}2506gitlab # [ 390.494438] gitaly[916]: time="2026-09-01T10:37:40.717Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QJTA6WPFWSKEM9GZPQVQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:37:40.704" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc2507gitlab # [ 390.496344] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:41 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 4662508gitlab # [ 390.497467] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:40.817Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"d5bd102fe1e8cb381032dff4","created_at":"2026-09-01T10:37:37.962Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.967Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-d5bd102fe1e8cb381032dff4: done: 0.16196 sec","job_status":"done","queue_duration_s":2.688265,"scheduling_latency_s":2.688289,"redis_calls":13,"redis_duration_s":0.003494,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000425,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001729,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000899,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000441,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.137836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000460673,"duration_s":0.16196,"completed_at":"2026-09-01T10:37:40.817Z","load_balancing_strategy":"primary","db_duration_s":0.007221,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2509gitlab # [ 390.547577] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:40.819Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"3b9e48d9e99d12d5c252a9a3","created_at":"2026-09-01T10:37:37.968Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.972Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-3b9e48d9e99d12d5c252a9a3: start","job_status":"start","queue_duration_s":2.846918,"scheduling_latency_s":2.846943}2510gitlab # [ 390.617909] nixbot[3225]: {"ts": "2026-09-01T10:37:41.587Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/d996dce18654c3c835478cedbe8d54682367b5ea \"HTTP/1.1 201 Created\""}2511gitlab # [ 390.619066] nixbot[3225]: {"ts": "2026-09-01T10:37:41.597Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-d996dce18654-4"}2512gitlab # [ 390.620080] nixbot[3225]: {"ts": "2026-09-01T10:37:41.723Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-d996dce18654-4", "argv0": "bwrap"}2513gitlab # [ 390.621049] gitaly[916]: time="2026-09-01T10:37:40.718Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QJTA6WPFWSKEM9GZPQVQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:37:40.704" grpc.time_ms=13.533 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc2514gitlab # [ 390.622041] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:40.874Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"3b9e48d9e99d12d5c252a9a3","created_at":"2026-09-01T10:37:37.968Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.972Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-3b9e48d9e99d12d5c252a9a3: done: 0.054143 sec","job_status":"done","queue_duration_s":2.846918,"scheduling_latency_s":2.846943,"redis_calls":10,"redis_duration_s":0.002686,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000463,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001448,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000775,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028085,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000458718,"duration_s":0.054143,"completed_at":"2026-09-01T10:37:40.873Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012092,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2515gitlab # [ 390.652211] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:40.875Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"6f78fa6a5bf224cd806ec1ae","created_at":"2026-09-01T10:37:37.973Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.979Z","job_size_bytes":19,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-6f78fa6a5bf224cd806ec1ae: start","job_status":"start","queue_duration_s":2.896343,"scheduling_latency_s":2.896365}2516gitlab # [ 390.673071] gitaly[916]: time="2026-09-01T10:37:40.823Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8QK0GS0JYDY1HCSMAYQKP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=196 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:37:40.813" grpc.time_ms=10 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc2517gitlab # [ 390.731814] gitaly[916]: time="2026-09-01T10:37:40.906Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QK2TMPE46XM7T9SQP49C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:37:40.897" pid=916 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2518gitlab # [ 390.732955] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:40.918Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"6f78fa6a5bf224cd806ec1ae","created_at":"2026-09-01T10:37:37.973Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.979Z","job_size_bytes":19,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-6f78fa6a5bf224cd806ec1ae: done: 0.043015 sec","job_status":"done","queue_duration_s":2.896343,"scheduling_latency_s":2.896365,"redis_calls":11,"redis_duration_s":0.004602,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000643,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003508,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000451,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025756,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000847594,"duration_s":0.043015,"completed_at":"2026-09-01T10:37:40.918Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007177,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2519gitlab # [ 390.757333] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:40.920Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"08a2d2a7a2ff46d5f09636f5","created_at":"2026-09-01T10:37:37.982Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.988Z","job_size_bytes":66,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-08a2d2a7a2ff46d5f09636f5: start","job_status":"start","queue_duration_s":2.932516,"scheduling_latency_s":2.932558}2520gitlab # [ 390.778296] gitaly[916]: time="2026-09-01T10:37:40.906Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QK2TMPE46XM7T9SQP49C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:37:40.897" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc2521gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea'2522gitlab # [ 390.794950] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:40.969Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"08a2d2a7a2ff46d5f09636f5","created_at":"2026-09-01T10:37:37.982Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.988Z","job_size_bytes":66,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-08a2d2a7a2ff46d5f09636f5: done: 0.048403 sec","job_status":"done","queue_duration_s":2.932516,"scheduling_latency_s":2.932558,"redis_calls":10,"redis_duration_s":0.003275,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000744,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001846,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000685,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000579682,"duration_s":0.048403,"completed_at":"2026-09-01T10:37:40.969Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008036,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2523gitlab # [ 390.854296] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:40.971Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"44eac48a98faf128ca0168ef","created_at":"2026-09-01T10:37:37.989Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.993Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-44eac48a98faf128ca0168ef: start","job_status":"start","queue_duration_s":2.977548,"scheduling_latency_s":2.977579}2524gitlab # [ 390.874110] gitaly[916]: time="2026-09-01T10:37:40.907Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QK2TMPE46XM7T9SQP49C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:37:40.897" grpc.time_ms=10.049 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc2525gitlab # [ 390.891427] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:41.033Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"44eac48a98faf128ca0168ef","created_at":"2026-09-01T10:37:37.989Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:37.993Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-44eac48a98faf128ca0168ef: done: 0.061843 sec","job_status":"done","queue_duration_s":2.977548,"scheduling_latency_s":2.977579,"redis_calls":10,"redis_duration_s":0.0046040000000000005,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001993,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001954,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000657,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03817,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000421282,"duration_s":0.061843,"completed_at":"2026-09-01T10:37:41.033Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008464,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2526gitlab # [ 390.971080] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:41.035Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"659a6e9200e3f57e3c7b3856","created_at":"2026-09-01T10:37:38.000Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:38.005Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-659a6e9200e3f57e3c7b3856: start","job_status":"start","queue_duration_s":3.030281,"scheduling_latency_s":3.030309}2527gitlab # [ 390.972102] gitaly[916]: time="2026-09-01T10:37:41.014Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=18 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8QK68BQ5VKRDW75QJY57D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:37:40.992" grpc.time_ms=22.034 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc2528gitlab # [ 390.979689] gitaly[916]: time="2026-09-01T10:37:41.094Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QK8TW472HM3MJG2MPHE1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:37:41.082" pid=916 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc2529gitlab # [ 390.991801] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:41.091Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"659a6e9200e3f57e3c7b3856","created_at":"2026-09-01T10:37:38.000Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:38.005Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-659a6e9200e3f57e3c7b3856: done: 0.056204 sec","job_status":"done","queue_duration_s":3.030281,"scheduling_latency_s":3.030309,"redis_calls":10,"redis_duration_s":0.004012,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000486,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002783,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000743,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001247644,"duration_s":0.056204,"completed_at":"2026-09-01T10:37:41.091Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009978,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2530gitlab # [ 391.069139] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:41.093Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"38923602fb80852bd47a3ff8","created_at":"2026-09-01T10:37:38.005Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:38.010Z","job_size_bytes":6,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-38923602fb80852bd47a3ff8: start","job_status":"start","queue_duration_s":3.083254,"scheduling_latency_s":3.083296}2531gitlab # [ 391.070090] gitaly[916]: time="2026-09-01T10:37:41.094Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1168 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QK8TW472HM3MJG2MPHE1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:37:41.082" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc2532gitlab # [ 391.083057] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:41.133Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"38923602fb80852bd47a3ff8","created_at":"2026-09-01T10:37:38.005Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:38.010Z","job_size_bytes":6,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-38923602fb80852bd47a3ff8: done: 0.040455 sec","job_status":"done","queue_duration_s":3.083254,"scheduling_latency_s":3.083296,"redis_calls":10,"redis_duration_s":0.003154,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000587,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001716,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000851,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024932,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000471568,"duration_s":0.040455,"completed_at":"2026-09-01T10:37:41.133Z","load_balancing_strategy":"primary","db_duration_s":0.004701,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2533gitlab # [ 391.142538] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:41.136Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"0dc8ed186724661d54fae87d","created_at":"2026-09-01T10:37:38.010Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:38.015Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-0dc8ed186724661d54fae87d: start","job_status":"start","queue_duration_s":3.120475,"scheduling_latency_s":3.120497}2534gitlab # [ 391.204261] gitaly[916]: time="2026-09-01T10:37:41.096Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1168 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QK8TW472HM3MJG2MPHE1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:37:41.082" grpc.time_ms=13.073 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc2535gitlab # [ 391.205627] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:41.201Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"0dc8ed186724661d54fae87d","created_at":"2026-09-01T10:37:38.010Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:38.015Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-0dc8ed186724661d54fae87d: done: 0.06499 sec","job_status":"done","queue_duration_s":3.120475,"scheduling_latency_s":3.120497,"redis_calls":10,"redis_duration_s":0.005333,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000887,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002894,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001552,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034673,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000504534,"duration_s":0.06499,"completed_at":"2026-09-01T10:37:41.200Z","load_balancing_strategy":"primary","db_duration_s":0.012445,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2536gitlab # [ 391.230243] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:41.203Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"6062dedb8d3e81c11c1b170a","created_at":"2026-09-01T10:37:38.021Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:38.025Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-6062dedb8d3e81c11c1b170a: start","job_status":"start","queue_duration_s":3.178121,"scheduling_latency_s":3.178227}2537gitlab # [ 391.250108] gitaly[916]: time="2026-09-01T10:37:41.204Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QKC23HRE8W1FN1G5XF62 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:11.002" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:41.203" grpc.time_ms=0.745 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2538gitlab # [ 391.265506] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:41.290Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"6062dedb8d3e81c11c1b170a","created_at":"2026-09-01T10:37:38.021Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:38.025Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-6062dedb8d3e81c11c1b170a: done: 0.087173 sec","job_status":"done","queue_duration_s":3.178121,"scheduling_latency_s":3.178227,"redis_calls":14,"redis_duration_s":0.006692,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00139,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002913,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002389,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043106,"duration_s":0.087173,"completed_at":"2026-09-01T10:37:41.290Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01399,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2539gitlab # [ 391.319300] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:41.292Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"de717dddc7e25d4572c05c1c","created_at":"2026-09-01T10:37:38.083Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:38.085Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-de717dddc7e25d4572c05c1c: start","job_status":"start","queue_duration_s":3.207073,"scheduling_latency_s":3.207116}2540gitlab # [ 391.338405] gitaly[916]: time="2026-09-01T10:37:41.249Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=6 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8QKC23HRE8W1FN1G5XF62 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-01T10:38:11.040" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:41.240" grpc.time_ms=8.473 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2541gitlab # [ 391.408193] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:41.719Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"de717dddc7e25d4572c05c1c","created_at":"2026-09-01T10:37:38.083Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:38.085Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3241,"job_status":"rescheduled","message":"PipelineProcessWorker JID-de717dddc7e25d4572c05c1c: rescheduled"}2542gitlab # [ 391.409174] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:41.728Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"de717dddc7e25d4572c05c1c","created_at":"2026-09-01T10:37:38.083Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-2df6e9102c6940b2","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:38.085Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-de717dddc7e25d4572c05c1c: done: 0.435552 sec","job_status":"done","queue_duration_s":3.207073,"scheduling_latency_s":3.207116,"gitaly_calls":3,"gitaly_duration_s":0.208923,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.018118,"redis_read_bytes":152,"redis_write_bytes":18010,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000634,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001042,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.00468,"redis_queues_read_bytes":30,"redis_queues_write_bytes":12309,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.006755,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000381,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000634,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.003358,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.042,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.009,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.009,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.024,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.166145,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.374590625,"duration_s":0.435552,"completed_at":"2026-09-01T10:37:41.728Z","load_balancing_strategy":"primary","db_duration_s":0.044752,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2543gitlab # [ 391.436864] gitaly[916]: time="2026-09-01T10:37:41.265Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8QKC23HRE8W1FN1G5XF62 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:37:51.186" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:41.258" grpc.time_ms=7.014 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2544gitlab # [ 391.453469] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:41.730Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"3f06b99820d4f0d0f0aae94a","created_at":"2026-09-01T10:37:38.258Z","trace_propagation_headers":{"sentry-trace":"9747fac109a543f6bf57052f34116f48-4d50ab2ecf844b2a","baggage":"sentry-trace_id=9747fac109a543f6bf57052f34116f48,sentry-release=Unknown"},"correlation_id":"01M1E8MZ3DZ3D1SWJBZTBSPN0K","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:38.263Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-3f06b99820d4f0d0f0aae94a: start","job_status":"start","queue_duration_s":3.466493,"scheduling_latency_s":3.466562}2545gitlab # [ 391.517690] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:41.792Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"3f06b99820d4f0d0f0aae94a","created_at":"2026-09-01T10:37:38.258Z","trace_propagation_headers":{"sentry-trace":"9747fac109a543f6bf57052f34116f48-4d50ab2ecf844b2a","baggage":"sentry-trace_id=9747fac109a543f6bf57052f34116f48,sentry-release=Unknown"},"correlation_id":"01M1E8MZ3DZ3D1SWJBZTBSPN0K","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:38.263Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-3f06b99820d4f0d0f0aae94a: done: 0.06222 sec","job_status":"done","queue_duration_s":3.466493,"scheduling_latency_s":3.466562,"redis_calls":19,"redis_duration_s":0.006396,"redis_read_bytes":22,"redis_write_bytes":4289,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001584,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000952,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2260,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002758,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001102,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043512,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.013303088000000001,"duration_s":0.06222,"completed_at":"2026-09-01T10:37:41.792Z","load_balancing_strategy":"primary","db_duration_s":0.007192,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2546gitlab # [ 391.526532] gitaly[916]: time="2026-09-01T10:37:41.301Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QKC23HRE8W1FN1G5XF62 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-01T10:37:51.230" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:37:41.301" grpc.time_ms=0.258 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2547gitlab # [ 391.539256] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:41.794Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","6aacd8b8ef73e9e7cb709123"],"class":"WebHooks::LogExecutionWorker","jid":"efb773d6d8aafaeaa2b6d72f","created_at":"2026-09-01T10:37:39.522Z","trace_propagation_headers":{"sentry-trace":"9747fac109a543f6bf57052f34116f48-1b10cb937d3347c4","baggage":"sentry-trace_id=9747fac109a543f6bf57052f34116f48,sentry-release=Unknown"},"correlation_id":"01M1E8MZ3DZ3D1SWJBZTBSPN0K","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:638996f0ccc25f7069cadb2594fc3e206ea32b510c68d0682c6ef029904b9f31","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:39.531Z","job_size_bytes":2592,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-efb773d6d8aafaeaa2b6d72f: start","job_status":"start","queue_duration_s":2.263127,"scheduling_latency_s":2.263174}2548gitlab # [ 391.559436] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:41.848Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","6aacd8b8ef73e9e7cb709123"],"class":"WebHooks::LogExecutionWorker","jid":"efb773d6d8aafaeaa2b6d72f","created_at":"2026-09-01T10:37:39.522Z","trace_propagation_headers":{"sentry-trace":"9747fac109a543f6bf57052f34116f48-1b10cb937d3347c4","baggage":"sentry-trace_id=9747fac109a543f6bf57052f34116f48,sentry-release=Unknown"},"correlation_id":"01M1E8MZ3DZ3D1SWJBZTBSPN0K","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:638996f0ccc25f7069cadb2594fc3e206ea32b510c68d0682c6ef029904b9f31","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:39.531Z","job_size_bytes":2592,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-efb773d6d8aafaeaa2b6d72f: done: 0.05445 sec","job_status":"done","queue_duration_s":2.263127,"scheduling_latency_s":2.263174,"redis_calls":10,"redis_duration_s":0.002346,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000429,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001455,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000462,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038069,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000451734,"duration_s":0.05445,"completed_at":"2026-09-01T10:37:41.848Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00725,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2549gitlab # [ 391.613043] gitaly[916]: time="2026-09-01T10:37:41.317Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QKC23HRE8W1FN1G5XF62 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-01T10:37:51.238" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:41.308" grpc.time_ms=8.842 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2550gitlab # [ 391.628423] gitaly[916]: time="2026-09-01T10:37:41.365Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8QKC23HRE8W1FN1G5XF62 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:37:41.364" grpc.time_ms=0.579 pid=916 span.kind=server system=grpc2551gitlab # [ 391.640043] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:41.851Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","db128f8054e797b151adc32f"],"class":"WebHooks::LogExecutionWorker","jid":"29455aec9cb59a1d5fd6878a","created_at":"2026-09-01T10:37:40.429Z","trace_propagation_headers":{"sentry-trace":"58684ea44cea4ffda557b00e4cb59895-4be40214322347da","baggage":"sentry-trace_id=58684ea44cea4ffda557b00e4cb59895,sentry-release=Unknown"},"correlation_id":"01M1E8NA0VZXEZDXQDHQ9N8F0K","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9056c37730a6d3b717e83d791374ee817f09edac38ec8389fb0f236e81bbf30b","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:40.436Z","job_size_bytes":2591,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-29455aec9cb59a1d5fd6878a: start","job_status":"start","queue_duration_s":1.414546,"scheduling_latency_s":1.414569}2552gitlab # [ 391.660050] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:41.882Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","db128f8054e797b151adc32f"],"class":"WebHooks::LogExecutionWorker","jid":"29455aec9cb59a1d5fd6878a","created_at":"2026-09-01T10:37:40.429Z","trace_propagation_headers":{"sentry-trace":"58684ea44cea4ffda557b00e4cb59895-4be40214322347da","baggage":"sentry-trace_id=58684ea44cea4ffda557b00e4cb59895,sentry-release=Unknown"},"correlation_id":"01M1E8NA0VZXEZDXQDHQ9N8F0K","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9056c37730a6d3b717e83d791374ee817f09edac38ec8389fb0f236e81bbf30b","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:40.436Z","job_size_bytes":2591,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-29455aec9cb59a1d5fd6878a: done: 0.031341 sec","job_status":"done","queue_duration_s":1.414546,"scheduling_latency_s":1.414569,"redis_calls":7,"redis_duration_s":0.001722,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0013,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000422,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031341,"completed_at":"2026-09-01T10:37:41.882Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004545,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2553gitlab # [ 391.711053] gitaly[916]: time="2026-09-01T10:37:41.394Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8QKC23HRE8W1FN1G5XF62 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:37:41.393" grpc.time_ms=0.437 pid=916 span.kind=server system=grpc2554gitlab # [ 391.722364] gitaly[916]: time="2026-09-01T10:37:41.418Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QKC23HRE8W1FN1G5XF62 diskcache=3f05ade6-264f-42ac-864e-9b6a4bb59243 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-01T10:37:51.251" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:37:41.321" pid=916 ref=refs/keep-around/d996dce18654c3c835478cedbe8d54682367b5ea remote_ip="2001:db8:1::1" revision=d996dce18654c3c835478cedbe8d54682367b5ea span.kind=server system=grpc username=root2555gitlab # [ 391.774137] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:41.884Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"f75f8cfec2b77c801e1c0cef","created_at":"2026-09-01T10:37:41.387Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-e08566d0ee214685","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.391Z","job_size_bytes":23,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-f75f8cfec2b77c801e1c0cef: start","job_status":"start","queue_duration_s":0.4925,"scheduling_latency_s":0.492524}2556gitlab # [ 391.775144] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:41.925Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"f75f8cfec2b77c801e1c0cef","created_at":"2026-09-01T10:37:41.387Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-e08566d0ee214685","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.391Z","job_size_bytes":23,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-f75f8cfec2b77c801e1c0cef: done: 0.041458 sec","job_status":"done","queue_duration_s":0.4925,"scheduling_latency_s":0.492524,"redis_calls":7,"redis_duration_s":0.00167,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001213,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000457,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041458,"completed_at":"2026-09-01T10:37:41.925Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006328,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2557gitlab # [ 391.814811] gitaly[916]: time="2026-09-01T10:37:41.419Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=54 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5080 command.oublock=8 command.real_time_ms=75 command.spawn_token_fork_ms=9 command.system_time_ms=31 command.user_time_ms=23 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QKC23HRE8W1FN1G5XF62 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-01T10:37:51.251" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:37:41.321" grpc.time_ms=96.915 pid=916 ref=refs/keep-around/d996dce18654c3c835478cedbe8d54682367b5ea remote_ip="2001:db8:1::1" revision=d996dce18654c3c835478cedbe8d54682367b5ea span.kind=server system=grpc username=root2558gitlab # [ 391.837289] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:41.927Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"b760837d3b54f5da2f370373","created_at":"2026-09-01T10:37:41.468Z","trace_propagation_headers":{"sentry-trace":"80010b1d93834996822f84345fcd5147-714bdb7da50b40c5","baggage":"sentry-trace_id=80010b1d93834996822f84345fcd5147,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8QKC23HRE8W1FN1G5XF62","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.481Z","job_size_bytes":65,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-b760837d3b54f5da2f370373: start","job_status":"start","queue_duration_s":0.445615,"scheduling_latency_s":0.445639}2559gitlab # [ 391.859764] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:41.954Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"b760837d3b54f5da2f370373","created_at":"2026-09-01T10:37:41.468Z","trace_propagation_headers":{"sentry-trace":"80010b1d93834996822f84345fcd5147-714bdb7da50b40c5","baggage":"sentry-trace_id=80010b1d93834996822f84345fcd5147,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8QKC23HRE8W1FN1G5XF62","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.481Z","job_size_bytes":65,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-b760837d3b54f5da2f370373: done: 0.027795 sec","job_status":"done","queue_duration_s":0.445615,"scheduling_latency_s":0.445639,"redis_calls":7,"redis_duration_s":0.001564,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001155,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000409,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027795,"completed_at":"2026-09-01T10:37:41.954Z","load_balancing_strategy":"primary","db_duration_s":0.004713,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2560gitlab # [ 391.914427] gitaly[916]: time="2026-09-01T10:37:41.451Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QKC23HRE8W1FN1G5XF62 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:11.248" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:41.449" grpc.time_ms=2.335 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2561gitlab # [ 391.930058] gitaly[916]: time="2026-09-01T10:37:41.474Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:37:41.473" grpc.time_ms=0.549 pid=916 span.kind=server system=grpc2562gitlab # [ 391.942075] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:41.956Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"97ce7cf459fd4e85077144e8","created_at":"2026-09-01T10:37:41.483Z","trace_propagation_headers":{"sentry-trace":"80010b1d93834996822f84345fcd5147-714bdb7da50b40c5","baggage":"sentry-trace_id=80010b1d93834996822f84345fcd5147,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8QKC23HRE8W1FN1G5XF62","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.494Z","job_size_bytes":65,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-97ce7cf459fd4e85077144e8: start","job_status":"start","queue_duration_s":0.462147,"scheduling_latency_s":0.462171}2563gitlab # [ 391.963081] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.008Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"97ce7cf459fd4e85077144e8","created_at":"2026-09-01T10:37:41.483Z","trace_propagation_headers":{"sentry-trace":"80010b1d93834996822f84345fcd5147-714bdb7da50b40c5","baggage":"sentry-trace_id=80010b1d93834996822f84345fcd5147,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8QKC23HRE8W1FN1G5XF62","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.494Z","job_size_bytes":65,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-97ce7cf459fd4e85077144e8: done: 0.051837 sec","job_status":"done","queue_duration_s":0.462147,"scheduling_latency_s":0.462171,"redis_calls":10,"redis_duration_s":0.0033599999999999997,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002266,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001094,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034957,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.051837,"completed_at":"2026-09-01T10:37:42.008Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011678,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2564gitlab # [ 392.016352] gitaly[916]: time="2026-09-01T10:37:41.515Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:37:41.514" grpc.time_ms=1.498 pid=916 span.kind=server system=grpc2565gitlab # [ 392.027850] gitaly[916]: time="2026-09-01T10:37:41.588Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:37:41.585" grpc.time_ms=1.871 pid=916 span.kind=server system=grpc2566gitlab # [ 392.081208] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.010Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"1ca779f07e8dbb0ac9903754","created_at":"2026-09-01T10:37:41.564Z","trace_propagation_headers":{"sentry-trace":"80010b1d93834996822f84345fcd5147-714bdb7da50b40c5","baggage":"sentry-trace_id=80010b1d93834996822f84345fcd5147,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8QKC23HRE8W1FN1G5XF62","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.569Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-1ca779f07e8dbb0ac9903754: start","job_status":"start","queue_duration_s":0.440857,"scheduling_latency_s":0.440881}2567gitlab # [ 392.082318] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.126Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"1ca779f07e8dbb0ac9903754","created_at":"2026-09-01T10:37:41.564Z","trace_propagation_headers":{"sentry-trace":"80010b1d93834996822f84345fcd5147-714bdb7da50b40c5","baggage":"sentry-trace_id=80010b1d93834996822f84345fcd5147,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8QKC23HRE8W1FN1G5XF62","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.569Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-1ca779f07e8dbb0ac9903754: done: 0.116222 sec","job_status":"done","queue_duration_s":0.440857,"scheduling_latency_s":0.440881,"gitaly_calls":2,"gitaly_duration_s":0.006184,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.00942,"redis_read_bytes":80,"redis_write_bytes":5121,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002531,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000874,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000239,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001514,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000318,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.00043,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000983,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.084983,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.086770525,"duration_s":0.116222,"completed_at":"2026-09-01T10:37:42.126Z","load_balancing_strategy":"primary","db_duration_s":0.022537,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2568gitlab # [ 392.120518] gitaly[916]: time="2026-09-01T10:37:41.598Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS diskcache=a2e6c397-eb93-465a-a19d-9066f2d60cf8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T10:38:11.400" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:41.401" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2569gitlab # [ 392.132131] gitaly[916]: time="2026-09-01T10:37:41.598Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=95 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7151 command.oublock=0 command.real_time_ms=190 command.spawn_token_fork_ms=4 command.system_time_ms=59 command.user_time_ms=36 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T10:38:11.400" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:41.401" grpc.time_ms=196.64 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2570gitlab # [ 392.190274] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.128Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"317e9e2c860337fb127e41b0","created_at":"2026-09-01T10:37:41.604Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-e08566d0ee214685","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.606Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-317e9e2c860337fb127e41b0: start","job_status":"start","queue_duration_s":0.522117,"scheduling_latency_s":0.522144}2571gitlab # [ 392.191382] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.156Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"317e9e2c860337fb127e41b0","created_at":"2026-09-01T10:37:41.604Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-e08566d0ee214685","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.606Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-317e9e2c860337fb127e41b0: done: 0.027364 sec","job_status":"done","queue_duration_s":0.522117,"scheduling_latency_s":0.522144,"redis_calls":7,"redis_duration_s":0.001843,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001303,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00054,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018532,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027364,"completed_at":"2026-09-01T10:37:42.156Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00405,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2572gitlab # [ 392.219961] gitaly[916]: time="2026-09-01T10:37:41.669Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-01T10:38:11.667" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:41.667" grpc.time_ms=1.92 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2573gitlab # [ 392.280281] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.157Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"7540f9d752d72b3e2ce9967a","created_at":"2026-09-01T10:37:41.614Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-e08566d0ee214685","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.616Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-7540f9d752d72b3e2ce9967a: start","job_status":"start","queue_duration_s":0.540947,"scheduling_latency_s":0.540971}2574gitlab # [ 392.281384] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.184Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"7540f9d752d72b3e2ce9967a","created_at":"2026-09-01T10:37:41.614Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-e08566d0ee214685","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.616Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-7540f9d752d72b3e2ce9967a: done: 0.026692 sec","job_status":"done","queue_duration_s":0.540947,"scheduling_latency_s":0.540971,"redis_calls":7,"redis_duration_s":0.001594,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001161,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000433,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017908,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026692,"completed_at":"2026-09-01T10:37:42.184Z","load_balancing_strategy":"primary","db_duration_s":0.004221,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2575gitlab # [ 392.347232] gitaly[916]: time="2026-09-01T10:37:41.706Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NAVTK64S37X9QW1ME0ZS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:11.704" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:41.705" grpc.time_ms=1.069 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2576gitlab # [ 392.348324] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.186Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"2ef178570939f0d0e72fe219","created_at":"2026-09-01T10:37:41.625Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-e08566d0ee214685","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.628Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-2ef178570939f0d0e72fe219: start","job_status":"start","queue_duration_s":0.557284,"scheduling_latency_s":0.557312}2577gitlab # [ 392.349223] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.216Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"2ef178570939f0d0e72fe219","created_at":"2026-09-01T10:37:41.625Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-e08566d0ee214685","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.628Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-2ef178570939f0d0e72fe219: done: 0.030693 sec","job_status":"done","queue_duration_s":0.557284,"scheduling_latency_s":0.557312,"redis_calls":10,"redis_duration_s":0.002533,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001467,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000637,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000429,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020571,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030693,"completed_at":"2026-09-01T10:37:42.216Z","load_balancing_strategy":"primary","db_duration_s":0.005041,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2578gitlab # [ 392.403340] gitaly[916]: time="2026-09-01T10:37:42.086Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8QKC23HRE8W1FN1G5XF62 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-01T10:38:12.084" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:42.085" grpc.time_ms=0.917 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2579gitlab # [ 392.420179] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.218Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"de5e59f6555a0def67980fd4","created_at":"2026-09-01T10:37:41.629Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-e08566d0ee214685","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.633Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-de5e59f6555a0def67980fd4: start","job_status":"start","queue_duration_s":0.584418,"scheduling_latency_s":0.584442}2580gitlab # [ 392.482847] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.242Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"de5e59f6555a0def67980fd4","created_at":"2026-09-01T10:37:41.629Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-e08566d0ee214685","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.633Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-de5e59f6555a0def67980fd4: done: 0.024384 sec","job_status":"done","queue_duration_s":0.584418,"scheduling_latency_s":0.584442,"redis_calls":7,"redis_duration_s":0.0016489999999999999,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001176,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000473,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024384,"completed_at":"2026-09-01T10:37:42.242Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003659,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2581gitlab # [ 392.539678] gitaly[916]: time="2026-09-01T10:37:42.116Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QKC23HRE8W1FN1G5XF62 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:12.114" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:42.115" grpc.time_ms=0.62 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2582gitlab # [ 392.540889] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.244Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ee7bdc086f160dd64c018bf0","created_at":"2026-09-01T10:37:41.634Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-e08566d0ee214685","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.638Z","job_size_bytes":19,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ee7bdc086f160dd64c018bf0: start","job_status":"start","queue_duration_s":0.605952,"scheduling_latency_s":0.605977}2583gitlab # [ 392.541745] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.267Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ee7bdc086f160dd64c018bf0","created_at":"2026-09-01T10:37:41.634Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-e08566d0ee214685","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.638Z","job_size_bytes":19,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ee7bdc086f160dd64c018bf0: done: 0.023024 sec","job_status":"done","queue_duration_s":0.605952,"scheduling_latency_s":0.605977,"redis_calls":8,"redis_duration_s":0.001861,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001441,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00042,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023024,"completed_at":"2026-09-01T10:37:42.267Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003103,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2584gitlab # [ 392.584117] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.269Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"1cd72ad8593eb23f64bb3026","created_at":"2026-09-01T10:37:41.639Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-e08566d0ee214685","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.641Z","job_size_bytes":66,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-1cd72ad8593eb23f64bb3026: start","job_status":"start","queue_duration_s":0.627382,"scheduling_latency_s":0.627408}2585gitlab # [ 392.605556] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.291Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"1cd72ad8593eb23f64bb3026","created_at":"2026-09-01T10:37:41.639Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-e08566d0ee214685","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.641Z","job_size_bytes":66,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-1cd72ad8593eb23f64bb3026: done: 0.022073 sec","job_status":"done","queue_duration_s":0.627382,"scheduling_latency_s":0.627408,"redis_calls":7,"redis_duration_s":0.00156,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001147,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000413,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014794,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022073,"completed_at":"2026-09-01T10:37:42.291Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002892,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2586gitlab # [ 392.659424] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.292Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"5b0ff95b5110bb6d472dd33a","created_at":"2026-09-01T10:37:41.642Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-e08566d0ee214685","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.644Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-5b0ff95b5110bb6d472dd33a: start","job_status":"start","queue_duration_s":0.648658,"scheduling_latency_s":0.648681}2587gitlab # [ 392.679830] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.316Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"5b0ff95b5110bb6d472dd33a","created_at":"2026-09-01T10:37:41.642Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-e08566d0ee214685","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.644Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-5b0ff95b5110bb6d472dd33a: done: 0.024161 sec","job_status":"done","queue_duration_s":0.648658,"scheduling_latency_s":0.648681,"redis_calls":7,"redis_duration_s":0.001471,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001092,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000379,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015951,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024161,"completed_at":"2026-09-01T10:37:42.316Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003842,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2588gitlab # [ 392.733192] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.318Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"6dd568f73f8be9b3b9a231e6","created_at":"2026-09-01T10:37:41.644Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-e08566d0ee214685","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.646Z","job_size_bytes":6,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-6dd568f73f8be9b3b9a231e6: start","job_status":"start","queue_duration_s":0.672273,"scheduling_latency_s":0.672299}2589gitlab # [ 392.752364] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.332Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"6dd568f73f8be9b3b9a231e6","created_at":"2026-09-01T10:37:41.644Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-e08566d0ee214685","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.646Z","job_size_bytes":6,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-6dd568f73f8be9b3b9a231e6: done: 0.014409 sec","job_status":"done","queue_duration_s":0.672273,"scheduling_latency_s":0.672299,"redis_calls":7,"redis_duration_s":0.0014449999999999999,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001058,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000387,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009477,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014409,"completed_at":"2026-09-01T10:37:42.332Z","load_balancing_strategy":"primary","db_duration_s":0.00065,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2590gitlab # [ 392.804547] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.334Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"2c8adaa927648d5b3a863ff1","created_at":"2026-09-01T10:37:41.646Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-e08566d0ee214685","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.648Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-2c8adaa927648d5b3a863ff1: start","job_status":"start","queue_duration_s":0.685977,"scheduling_latency_s":0.686001}2591gitlab # [ 392.822954] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.357Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"2c8adaa927648d5b3a863ff1","created_at":"2026-09-01T10:37:41.646Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-e08566d0ee214685","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.648Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-2c8adaa927648d5b3a863ff1: done: 0.023367 sec","job_status":"done","queue_duration_s":0.685977,"scheduling_latency_s":0.686001,"redis_calls":7,"redis_duration_s":0.0015739999999999999,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001137,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000437,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016233,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023367,"completed_at":"2026-09-01T10:37:42.357Z","load_balancing_strategy":"primary","db_duration_s":0.003074,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2592gitlab # [ 392.873384] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.358Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"91d512d690d0205e4009e04f","created_at":"2026-09-01T10:37:41.720Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-e08566d0ee214685","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.722Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-91d512d690d0205e4009e04f: start","job_status":"start","queue_duration_s":0.636755,"scheduling_latency_s":0.63678}2593gitlab # [ 392.892078] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.383Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"91d512d690d0205e4009e04f","created_at":"2026-09-01T10:37:41.720Z","trace_propagation_headers":{"sentry-trace":"4756735d26b54bcfb14b9b2a8922e8e9-e08566d0ee214685","baggage":"sentry-trace_id=4756735d26b54bcfb14b9b2a8922e8e9,sentry-release=Unknown"},"correlation_id":"01M1E8NAVTK64S37X9QW1ME0ZS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.722Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-91d512d690d0205e4009e04f: done: 0.024501 sec","job_status":"done","queue_duration_s":0.636755,"scheduling_latency_s":0.63678,"redis_calls":8,"redis_duration_s":0.001729,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001324,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000405,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017082,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024501,"completed_at":"2026-09-01T10:37:42.383Z","load_balancing_strategy":"primary","db_duration_s":0.003316,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2594gitlab # [ 392.944208] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.384Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"39717b11a7f531dc6e824299","created_at":"2026-09-01T10:37:41.769Z","trace_propagation_headers":{"sentry-trace":"9747fac109a543f6bf57052f34116f48-016a6f74191843c8","baggage":"sentry-trace_id=9747fac109a543f6bf57052f34116f48,sentry-release=Unknown"},"correlation_id":"01M1E8MZ3DZ3D1SWJBZTBSPN0K","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.778Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-39717b11a7f531dc6e824299: start","job_status":"start","queue_duration_s":0.6067,"scheduling_latency_s":0.606725}2595gitlab # [ 392.962174] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.464Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"39717b11a7f531dc6e824299","created_at":"2026-09-01T10:37:41.769Z","trace_propagation_headers":{"sentry-trace":"9747fac109a543f6bf57052f34116f48-016a6f74191843c8","baggage":"sentry-trace_id=9747fac109a543f6bf57052f34116f48,sentry-release=Unknown"},"correlation_id":"01M1E8MZ3DZ3D1SWJBZTBSPN0K","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:41.778Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-39717b11a7f531dc6e824299: done: 0.079285 sec","job_status":"done","queue_duration_s":0.6067,"scheduling_latency_s":0.606725,"redis_calls":11,"redis_duration_s":0.0027530000000000002,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000484,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001814,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000455,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000336914,"duration_s":0.079285,"completed_at":"2026-09-01T10:37:42.463Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01632,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2596gitlab # [ 393.016951] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.466Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"77ed1801b112cfe0e70e62ef","created_at":"2026-09-01T10:37:42.069Z","trace_propagation_headers":{"sentry-trace":"80010b1d93834996822f84345fcd5147-69e083d5032041e5","baggage":"sentry-trace_id=80010b1d93834996822f84345fcd5147,sentry-release=Unknown"},"correlation_id":"01M1E8QKC23HRE8W1FN1G5XF62","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:42.071Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-77ed1801b112cfe0e70e62ef: start","job_status":"start","queue_duration_s":0.394366,"scheduling_latency_s":0.394391}2597gitlab # [ 393.036638] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:42.498Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"77ed1801b112cfe0e70e62ef","created_at":"2026-09-01T10:37:42.069Z","trace_propagation_headers":{"sentry-trace":"80010b1d93834996822f84345fcd5147-69e083d5032041e5","baggage":"sentry-trace_id=80010b1d93834996822f84345fcd5147,sentry-release=Unknown"},"correlation_id":"01M1E8QKC23HRE8W1FN1G5XF62","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:42.071Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-77ed1801b112cfe0e70e62ef: done: 0.032799 sec","job_status":"done","queue_duration_s":0.394366,"scheduling_latency_s":0.394391,"redis_calls":7,"redis_duration_s":0.001627,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001192,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000435,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023554,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032799,"completed_at":"2026-09-01T10:37:42.498Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005098,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2598gitlab # [ 393.371068] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 9197943164 wd_nsec: 91979400162599gitlab # [ 393.155603] nixbot[3225]: INFO: ::ffff:127.0.0.1:38570 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 200 OK2600gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d996dce18654c3c835478cedbe8d54682367b5ea', in 2.30 seconds)2601[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'd996dce18654c3c835478cedbe8d54682367b5ea', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-09-01T10:37:41.115810Z', 'started_at': '2026-09-01T10:37:41.594678Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}]2602(finished: subtest: push delivers the webhook, in 78.55 seconds)2603subtest: eval and build post commit statuses2604gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses2605gitlab # [ 393.203511] nix-daemon[3485]: accepted connection from pid 4109, user nixbot2606gitlab # [ 393.263413] gitaly[916]: time="2026-09-01T10:37:44.420Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QPHAMBBPGAR2Y9M6NS6D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:14.219" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:44.420" grpc.time_ms=0.591 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2607gitlab # [ 393.310085] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:44 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1052608gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses, in 0.13 seconds)2609[{'id': 5, 'sha': 'd996dce18654c3c835478cedbe8d54682367b5ea', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-01T10:37:41.557Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2610gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses2611gitlab # [ 394.375793] gitaly[916]: time="2026-09-01T10:37:45.533Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QQMGTX0NKGNQSQ99V8DE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:15.332" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:45.532" grpc.time_ms=0.626 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2612gitlab # [ 394.402400] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:45 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 702613gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses, in 0.09 seconds)2614[{'id': 5, 'sha': 'd996dce18654c3c835478cedbe8d54682367b5ea', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-01T10:37:41.557Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2615gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses2616gitlab # [ 395.461579] gitaly[916]: time="2026-09-01T10:37:46.619Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QRPM76PF39NH4Q96R13K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:16.417" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:46.618" grpc.time_ms=0.581 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2617gitlab # [ 395.483453] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:46 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 602618gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses, in 0.08 seconds)2619[{'id': 5, 'sha': 'd996dce18654c3c835478cedbe8d54682367b5ea', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-01T10:37:41.557Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2620gitlab # [ 395.799079] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:46.956Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"1dc537ad5f97682fd161e865","created_at":"2026-09-01T10:37:46.948Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c80eb577d6482674bb25f8f4639c4f86","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-01T10:37:46.954Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-1dc537ad5f97682fd161e865: start","job_status":"start","queue_duration_s":0.00183,"scheduling_latency_s":0.001862}2621gitlab # [ 395.855913] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:47.013Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2622gitlab # [ 395.872602] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:47.030Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"1dc537ad5f97682fd161e865","created_at":"2026-09-01T10:37:46.948Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c80eb577d6482674bb25f8f4639c4f86","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-01T10:37:46.954Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-1dc537ad5f97682fd161e865: done: 0.073839 sec","job_status":"done","queue_duration_s":0.00183,"scheduling_latency_s":0.001862,"redis_calls":9,"redis_duration_s":0.011846,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001295,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008363,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002188,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017556,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.073839,"completed_at":"2026-09-01T10:37:47.030Z","load_balancing_strategy":"primary","db_duration_s":0.004389,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2623gitlab # [ 395.924914] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:47.040Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"6fac95f3511301b6ac9dfb42","created_at":"2026-09-01T10:37:46.986Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d95a84dd9ff70728df370ddc77e6e4c3","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.002Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-6fac95f3511301b6ac9dfb42: start","job_status":"start","queue_duration_s":0.038244,"scheduling_latency_s":0.03827}2624gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses2625gitlab # [ 396.552853] gitaly[916]: time="2026-09-01T10:37:47.710Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QSRJ9MWZXQYT593QW6GE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:17.509" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:47.709" grpc.time_ms=0.597 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2626gitlab # [ 396.585796] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:47.743Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"6fac95f3511301b6ac9dfb42","created_at":"2026-09-01T10:37:46.986Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d95a84dd9ff70728df370ddc77e6e4c3","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.002Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-6fac95f3511301b6ac9dfb42: done: 0.702749 sec","job_status":"done","queue_duration_s":0.038244,"scheduling_latency_s":0.03827,"redis_calls":564,"redis_duration_s":0.365118,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001073,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00182,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.362225,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.37104,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.702749,"completed_at":"2026-09-01T10:37:47.743Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000752,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2627gitlab # [ 396.589538] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:47 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 802628gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses, in 0.16 seconds)2629[{'id': 5, 'sha': 'd996dce18654c3c835478cedbe8d54682367b5ea', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-01T10:37:41.557Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2630gitlab # [ 396.645392] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:47.802Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"dc4c02807faa65925c28cab7","created_at":"2026-09-01T10:37:47.008Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"903e3898565fc96576619c0ee5468171","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.024Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-dc4c02807faa65925c28cab7: start","job_status":"start","queue_duration_s":0.777401,"scheduling_latency_s":0.777434}2631gitlab # [ 396.697272] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:47.854Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"dc4c02807faa65925c28cab7","created_at":"2026-09-01T10:37:47.008Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"903e3898565fc96576619c0ee5468171","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.024Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-dc4c02807faa65925c28cab7: done: 0.052856 sec","job_status":"done","queue_duration_s":0.777401,"scheduling_latency_s":0.777434,"redis_calls":11,"redis_duration_s":0.003099,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000422,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001293,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001384,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000641981,"exclusive_lock_hold_duration_s":0.011021792,"duration_s":0.052856,"completed_at":"2026-09-01T10:37:47.854Z","load_balancing_strategy":"primary","db_duration_s":0.006186,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2632gitlab # [ 396.754289] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:47.911Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"833b91d2a5f7bccbb58b0ce7","created_at":"2026-09-01T10:37:47.038Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.063Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-833b91d2a5f7bccbb58b0ce7: start","job_status":"start","queue_duration_s":0.848499,"scheduling_latency_s":0.848546}2633gitlab # [ 396.847839] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.005Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"833b91d2a5f7bccbb58b0ce7","created_at":"2026-09-01T10:37:47.038Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.063Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-833b91d2a5f7bccbb58b0ce7: done: 0.093215 sec","job_status":"done","queue_duration_s":0.848499,"scheduling_latency_s":0.848546,"redis_calls":62,"redis_duration_s":0.014366,"redis_read_bytes":110,"redis_write_bytes":41440,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00071,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_calls":2,"redis_queues_duration_s":0.00082,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.012031,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000805,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074357,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.093215,"completed_at":"2026-09-01T10:37:48.004Z","load_balancing_strategy":"primary","db_duration_s":0.000667,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2634gitlab # [ 396.849302] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.007Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"f7eeb5ae24596136f1fb1334","created_at":"2026-09-01T10:37:47.071Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd68d2a927c0a302c731cdf6425f9d01","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.079Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-f7eeb5ae24596136f1fb1334: start","job_status":"start","queue_duration_s":0.927792,"scheduling_latency_s":0.927829}2635gitlab # [ 396.884498] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.041Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"f7eeb5ae24596136f1fb1334","created_at":"2026-09-01T10:37:47.071Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd68d2a927c0a302c731cdf6425f9d01","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.079Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-f7eeb5ae24596136f1fb1334: done: 0.034753 sec","job_status":"done","queue_duration_s":0.927792,"scheduling_latency_s":0.927829,"redis_calls":9,"redis_duration_s":0.0027410000000000004,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000586,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001587,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000568,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019502,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034753,"completed_at":"2026-09-01T10:37:48.041Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006108,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2636gitlab # [ 396.927440] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.044Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"0081c4bc50e3c67d0ac81c1a","created_at":"2026-09-01T10:37:47.090Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"65fde164f6f51e2c17df5927953d2866","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.098Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-0081c4bc50e3c67d0ac81c1a: start","job_status":"start","queue_duration_s":0.945115,"scheduling_latency_s":0.945144}2637gitlab # [ 396.948159] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.072Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"0081c4bc50e3c67d0ac81c1a","created_at":"2026-09-01T10:37:47.090Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"65fde164f6f51e2c17df5927953d2866","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.098Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-0081c4bc50e3c67d0ac81c1a: done: 0.028482 sec","job_status":"done","queue_duration_s":0.945115,"scheduling_latency_s":0.945144,"redis_calls":9,"redis_duration_s":0.002314,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00053,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001288,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000496,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01512,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.028482,"completed_at":"2026-09-01T10:37:48.072Z","load_balancing_strategy":"primary","db_duration_s":0.006354,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2638gitlab # [ 397.046208] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.074Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"8308111c321ef118be776aac","created_at":"2026-09-01T10:37:47.106Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c66b4e0b93af9212bb62ed96aa53d461","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.114Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-8308111c321ef118be776aac: start","job_status":"start","queue_duration_s":0.959975,"scheduling_latency_s":0.96}2639gitlab # [ 397.047366] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.085Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2640gitlab # [ 397.048208] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.092Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"8308111c321ef118be776aac","created_at":"2026-09-01T10:37:47.106Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c66b4e0b93af9212bb62ed96aa53d461","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.114Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-8308111c321ef118be776aac: done: 0.017993 sec","job_status":"done","queue_duration_s":0.959975,"scheduling_latency_s":0.96,"redis_calls":9,"redis_duration_s":0.002085,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000402,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00125,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000433,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010694,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017993,"completed_at":"2026-09-01T10:37:48.092Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000581,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2641gitlab # [ 397.101348] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.093Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"47d0dcc35684208093140f1d","created_at":"2026-09-01T10:37:47.127Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ec3aa07c5cb7abfc42f52268549111b","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.136Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-47d0dcc35684208093140f1d: start","job_status":"start","queue_duration_s":0.956951,"scheduling_latency_s":0.956974}2642gitlab # [ 397.180407] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.116Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"47d0dcc35684208093140f1d","created_at":"2026-09-01T10:37:47.127Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ec3aa07c5cb7abfc42f52268549111b","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.136Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-47d0dcc35684208093140f1d: done: 0.022867 sec","job_status":"done","queue_duration_s":0.956951,"scheduling_latency_s":0.956974,"redis_calls":11,"redis_duration_s":0.003853,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000426,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001248,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002179,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00053247,"exclusive_lock_hold_duration_s":0.002275708,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.022867,"completed_at":"2026-09-01T10:37:48.116Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001431,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2643gitlab # [ 397.184875] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.117Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"ffae6818bc4ccd8ac5f9c232","created_at":"2026-09-01T10:37:47.162Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d7ce11e66ead7dd6b4f638f8d9403508","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.170Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-ffae6818bc4ccd8ac5f9c232: start","job_status":"start","queue_duration_s":0.947565,"scheduling_latency_s":0.947587}2644gitlab # [ 397.206539] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.126Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2645gitlab # [ 397.211895] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.132Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"ffae6818bc4ccd8ac5f9c232","created_at":"2026-09-01T10:37:47.162Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d7ce11e66ead7dd6b4f638f8d9403508","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.170Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-ffae6818bc4ccd8ac5f9c232: done: 0.015038 sec","job_status":"done","queue_duration_s":0.947565,"scheduling_latency_s":0.947587,"redis_calls":8,"redis_duration_s":0.002071,"redis_read_bytes":133,"redis_write_bytes":867,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000296,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":101,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00137,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000405,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00852,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015038,"completed_at":"2026-09-01T10:37:48.132Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2646gitlab # [ 397.307549] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.134Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"e4cc4f86e843ce344aeffff5","created_at":"2026-09-01T10:37:47.177Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8db1f8a6158be12f4b9deff7d211d1c4","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.180Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-e4cc4f86e843ce344aeffff5: start","job_status":"start","queue_duration_s":0.953653,"scheduling_latency_s":0.953677}2647gitlab # [ 397.308686] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.148Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"e4cc4f86e843ce344aeffff5","created_at":"2026-09-01T10:37:47.177Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8db1f8a6158be12f4b9deff7d211d1c4","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.180Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-e4cc4f86e843ce344aeffff5: done: 0.013653 sec","job_status":"done","queue_duration_s":0.953653,"scheduling_latency_s":0.953677,"redis_calls":7,"redis_duration_s":0.001489,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001111,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000378,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008431,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013653,"completed_at":"2026-09-01T10:37:48.148Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000828,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2648gitlab # [ 397.341752] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.149Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"9760c3013c725c0807a48af0","created_at":"2026-09-01T10:37:47.196Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d961d7929bf952c1cf4ae9aeaf928cae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.200Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-9760c3013c725c0807a48af0: start","job_status":"start","queue_duration_s":0.94924,"scheduling_latency_s":0.949262}2649gitlab # [ 397.359278] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.176Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"9760c3013c725c0807a48af0","created_at":"2026-09-01T10:37:47.196Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d961d7929bf952c1cf4ae9aeaf928cae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.200Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-9760c3013c725c0807a48af0: done: 0.02632 sec","job_status":"done","queue_duration_s":0.94924,"scheduling_latency_s":0.949262,"redis_calls":9,"redis_duration_s":0.001894,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015464,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000461512,"exclusive_lock_hold_duration_s":0.012820065,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.02632,"completed_at":"2026-09-01T10:37:48.176Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006776,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2650gitlab # [ 397.415317] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.177Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"bf68434571f48f8f245abde2","created_at":"2026-09-01T10:37:47.209Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba7d06fd2eef588b9778c36a5b1b7b60","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.212Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-bf68434571f48f8f245abde2: start","job_status":"start","queue_duration_s":0.965067,"scheduling_latency_s":0.965087}2651gitlab # [ 397.431924] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.198Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"bf68434571f48f8f245abde2","created_at":"2026-09-01T10:37:47.209Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba7d06fd2eef588b9778c36a5b1b7b60","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.212Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-bf68434571f48f8f245abde2: done: 0.020999 sec","job_status":"done","queue_duration_s":0.965067,"scheduling_latency_s":0.965087,"redis_calls":14,"redis_duration_s":0.002839,"redis_read_bytes":16,"redis_write_bytes":4026,"redis_queues_calls":2,"redis_queues_duration_s":0.000327,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001912,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.0006,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014993,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020999,"completed_at":"2026-09-01T10:37:48.198Z","load_balancing_strategy":"primary","db_duration_s":0.001065,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2652gitlab # [ 397.484106] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.200Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"375fc8c7964a93d4f3adb06a","created_at":"2026-09-01T10:37:47.223Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee5874d63a1019e79da1d05e2c50a8db","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.228Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-375fc8c7964a93d4f3adb06a: start","job_status":"start","queue_duration_s":0.971742,"scheduling_latency_s":0.971768}2653gitlab # [ 397.501784] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.207Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2654gitlab # [ 397.505355] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.212Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"375fc8c7964a93d4f3adb06a","created_at":"2026-09-01T10:37:47.223Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee5874d63a1019e79da1d05e2c50a8db","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.228Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-375fc8c7964a93d4f3adb06a: done: 0.012077 sec","job_status":"done","queue_duration_s":0.971742,"scheduling_latency_s":0.971768,"redis_calls":7,"redis_duration_s":0.001401,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000401,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007853,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012077,"completed_at":"2026-09-01T10:37:48.212Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2655gitlab # [ 397.554797] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.213Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"35d03df0050202228dcbd3c1","created_at":"2026-09-01T10:37:47.238Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cbec053bf95c33f89b635547230ae20a","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.244Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-35d03df0050202228dcbd3c1: start","job_status":"start","queue_duration_s":0.968968,"scheduling_latency_s":0.96899}2656gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses2657gitlab # [ 397.572552] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.231Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"35d03df0050202228dcbd3c1","created_at":"2026-09-01T10:37:47.238Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cbec053bf95c33f89b635547230ae20a","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.244Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-35d03df0050202228dcbd3c1: done: 0.018119 sec","job_status":"done","queue_duration_s":0.968968,"scheduling_latency_s":0.96899,"redis_calls":10,"redis_duration_s":0.002428,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001514,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000914,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.004091023,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.018119,"completed_at":"2026-09-01T10:37:48.231Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001956,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2658gitlab # [ 397.649431] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.233Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"33c38c32263878f9efd3a4b7","created_at":"2026-09-01T10:37:47.262Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9c056c2c7a297863835644c2b89272af","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.266Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-33c38c32263878f9efd3a4b7: start","job_status":"start","queue_duration_s":0.966881,"scheduling_latency_s":0.966904}2659gitlab # [ 397.650590] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.261Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"33c38c32263878f9efd3a4b7","created_at":"2026-09-01T10:37:47.262Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9c056c2c7a297863835644c2b89272af","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.266Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-33c38c32263878f9efd3a4b7: done: 0.02761 sec","job_status":"done","queue_duration_s":0.966881,"scheduling_latency_s":0.966904,"redis_calls":9,"redis_duration_s":0.0020819999999999996,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001107,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000975,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014253,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000596445,"exclusive_lock_hold_duration_s":0.010000992,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.02761,"completed_at":"2026-09-01T10:37:48.260Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009094,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2660gitlab # [ 397.703520] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.262Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"8542b308b085d42723b8cc8a","created_at":"2026-09-01T10:37:47.348Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b194fdd8cc1364916469ceb6cd8ec1e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.352Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-8542b308b085d42723b8cc8a: start","job_status":"start","queue_duration_s":0.909973,"scheduling_latency_s":0.910012}2661gitlab # [ 397.770854] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.275Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"8542b308b085d42723b8cc8a","created_at":"2026-09-01T10:37:47.348Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b194fdd8cc1364916469ceb6cd8ec1e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.352Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-8542b308b085d42723b8cc8a: done: 0.013175 sec","job_status":"done","queue_duration_s":0.909973,"scheduling_latency_s":0.910012,"redis_calls":7,"redis_duration_s":0.001429,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001035,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000394,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008619,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013175,"completed_at":"2026-09-01T10:37:48.275Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2662gitlab # [ 397.774119] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.277Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"0ab1e7eb8d734c435cc15a07","created_at":"2026-09-01T10:37:47.368Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"510c4e56cf6efd16cfd081ac446fde3d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.371Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-0ab1e7eb8d734c435cc15a07: start","job_status":"start","queue_duration_s":0.905268,"scheduling_latency_s":0.905291}2663gitlab # [ 397.793323] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.284Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2664gitlab # [ 397.799159] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.288Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"0ab1e7eb8d734c435cc15a07","created_at":"2026-09-01T10:37:47.368Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"510c4e56cf6efd16cfd081ac446fde3d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.371Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-0ab1e7eb8d734c435cc15a07: done: 0.01186 sec","job_status":"done","queue_duration_s":0.905268,"scheduling_latency_s":0.905291,"redis_calls":7,"redis_duration_s":0.0014199999999999998,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000437,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007625,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01186,"completed_at":"2026-09-01T10:37:48.288Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2665gitlab # [ 397.890070] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.290Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"b2759f7cd6145b6cb6f19a45","created_at":"2026-09-01T10:37:47.386Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0d162b0fc6407ba3dc30eba08f7da7ae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-01T10:37:47.389Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-b2759f7cd6145b6cb6f19a45: start","job_status":"start","queue_duration_s":0.900579,"scheduling_latency_s":0.9006}2666gitlab # [ 397.890914] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.297Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2667gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses, in 0.33 seconds)2668[{'id': 5, 'sha': 'd996dce18654c3c835478cedbe8d54682367b5ea', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-01T10:37:41.557Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2669gitlab # [ 397.893114] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.302Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"b2759f7cd6145b6cb6f19a45","created_at":"2026-09-01T10:37:47.386Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0d162b0fc6407ba3dc30eba08f7da7ae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-01T10:37:47.389Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-b2759f7cd6145b6cb6f19a45: done: 0.011838 sec","job_status":"done","queue_duration_s":0.900579,"scheduling_latency_s":0.9006,"redis_calls":7,"redis_duration_s":0.0015639999999999999,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00115,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000414,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007655,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011838,"completed_at":"2026-09-01T10:37:48.302Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2670gitlab # [ 397.923257] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.303Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"383424e506281e110df723b5","created_at":"2026-09-01T10:37:47.403Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85de03610b5bd03e456e616f4e558b96","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.406Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-383424e506281e110df723b5: start","job_status":"start","queue_duration_s":0.896773,"scheduling_latency_s":0.896798}2671gitlab # [ 397.940762] nix-daemon[3485]: accepted connection from pid 4157, user nixbot2672gitlab # [ 397.942431] gitaly[916]: time="2026-09-01T10:37:49.098Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QV3WQCDMJ4D3V8EVJH2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:19.096" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:49.097" grpc.time_ms=1.073 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2673gitlab # [ 397.957461] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.495Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"383424e506281e110df723b5","created_at":"2026-09-01T10:37:47.403Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85de03610b5bd03e456e616f4e558b96","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.406Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-383424e506281e110df723b5: done: 0.192373 sec","job_status":"done","queue_duration_s":0.896773,"scheduling_latency_s":0.896798,"redis_calls":9,"redis_duration_s":0.0023740000000000002,"redis_read_bytes":7,"redis_write_bytes":987,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001163,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001211,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":129,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":129,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.031,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.151636,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000637511,"exclusive_lock_hold_duration_s":0.177071388,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.192373,"completed_at":"2026-09-01T10:37:48.495Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.05004,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2674gitlab # [ 398.010975] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.497Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"2b99c9f2d97efafa407bf20f","created_at":"2026-09-01T10:37:47.413Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b7e0a5969cc0105e3abea32dd0f7ce9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.417Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-2b99c9f2d97efafa407bf20f: start","job_status":"start","queue_duration_s":1.080171,"scheduling_latency_s":1.080196}2675gitlab # [ 398.031156] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.510Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"2b99c9f2d97efafa407bf20f","created_at":"2026-09-01T10:37:47.413Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b7e0a5969cc0105e3abea32dd0f7ce9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.417Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-2b99c9f2d97efafa407bf20f: done: 0.013467 sec","job_status":"done","queue_duration_s":1.080171,"scheduling_latency_s":1.080196,"redis_calls":7,"redis_duration_s":0.001377,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001009,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000368,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013467,"completed_at":"2026-09-01T10:37:48.510Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2676gitlab # [ 398.084885] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.512Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"8b6a03640fc0848cc62e9b67","created_at":"2026-09-01T10:37:47.427Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3d3ba6a91a4ac0c03cb12ac92a3f0e2d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-01T10:37:47.431Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-8b6a03640fc0848cc62e9b67: start","job_status":"start","queue_duration_s":1.081125,"scheduling_latency_s":1.081153}2677gitlab # [ 398.102068] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 722678gitlab # [ 398.160971] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.534Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"8b6a03640fc0848cc62e9b67","created_at":"2026-09-01T10:37:47.427Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-1aa32b7f94f44132","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3d3ba6a91a4ac0c03cb12ac92a3f0e2d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-01T10:37:47.431Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-8b6a03640fc0848cc62e9b67: done: 0.022053 sec","job_status":"done","queue_duration_s":1.081125,"scheduling_latency_s":1.081153,"redis_calls":7,"redis_duration_s":0.001851,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001175,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000676,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015414,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022053,"completed_at":"2026-09-01T10:37:48.534Z","load_balancing_strategy":"primary","db_duration_s":0.001662,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2679gitlab # [ 398.166318] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.535Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd19969d9c86f9e344e2b772","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd19969d9c86f9e344e2b772: start","job_status":"start","queue_duration_s":0.541783,"scheduling_latency_s":0.541807}2680gitlab # [ 398.175073] nix-daemon[3485]: accepted connection from pid 4109, user nixbot2681gitlab # [ 398.176113] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.555Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd19969d9c86f9e344e2b772","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd19969d9c86f9e344e2b772: done: 0.019763 sec","job_status":"done","queue_duration_s":0.541783,"scheduling_latency_s":0.541807,"redis_calls":13,"redis_duration_s":0.003692,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001033,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002659,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013373,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019763,"completed_at":"2026-09-01T10:37:48.555Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2682gitlab # [ 398.229232] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.557Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5751cf38d8aff0964baebdd3","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5751cf38d8aff0964baebdd3: start","job_status":"start","queue_duration_s":0.562966,"scheduling_latency_s":0.562991}2683gitlab # [ 398.246715] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.575Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5751cf38d8aff0964baebdd3","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5751cf38d8aff0964baebdd3: done: 0.01847 sec","job_status":"done","queue_duration_s":0.562966,"scheduling_latency_s":0.562991,"redis_calls":13,"redis_duration_s":0.002689,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000956,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001733,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012185,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01847,"completed_at":"2026-09-01T10:37:48.575Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2684gitlab # [ 398.296968] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.577Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5b312be231f5d8cdd59e497","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5b312be231f5d8cdd59e497: start","job_status":"start","queue_duration_s":0.583326,"scheduling_latency_s":0.583358}2685gitlab # [ 398.313116] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.596Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5b312be231f5d8cdd59e497","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5b312be231f5d8cdd59e497: done: 0.019387 sec","job_status":"done","queue_duration_s":0.583326,"scheduling_latency_s":0.583358,"redis_calls":13,"redis_duration_s":0.002714,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000947,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001767,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013261,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019387,"completed_at":"2026-09-01T10:37:48.596Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2686gitlab # [ 398.404169] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.598Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf6609874cdb69e211daea04","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf6609874cdb69e211daea04: start","job_status":"start","queue_duration_s":0.60439,"scheduling_latency_s":0.604435}2687gitlab # [ 398.405351] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.616Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf6609874cdb69e211daea04","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf6609874cdb69e211daea04: done: 0.01828 sec","job_status":"done","queue_duration_s":0.60439,"scheduling_latency_s":0.604435,"redis_calls":13,"redis_duration_s":0.002511,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000833,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001678,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011325,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01828,"completed_at":"2026-09-01T10:37:48.616Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2688gitlab # [ 398.440075] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.618Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61a246cb96308c7dfba55594","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61a246cb96308c7dfba55594: start","job_status":"start","queue_duration_s":0.624338,"scheduling_latency_s":0.624366}2689gitlab # [ 398.458050] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.638Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61a246cb96308c7dfba55594","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61a246cb96308c7dfba55594: done: 0.019812 sec","job_status":"done","queue_duration_s":0.624338,"scheduling_latency_s":0.624366,"redis_calls":13,"redis_duration_s":0.003169,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001076,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013148,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019812,"completed_at":"2026-09-01T10:37:48.638Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2690gitlab # [ 398.550675] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.640Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e2033b2f3a9380811bfe8bf","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e2033b2f3a9380811bfe8bf: start","job_status":"start","queue_duration_s":0.646,"scheduling_latency_s":0.646034}2691gitlab # [ 398.552081] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.661Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e2033b2f3a9380811bfe8bf","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e2033b2f3a9380811bfe8bf: done: 0.021761 sec","job_status":"done","queue_duration_s":0.646,"scheduling_latency_s":0.646034,"redis_calls":13,"redis_duration_s":0.003017,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000997,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00202,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015193,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021761,"completed_at":"2026-09-01T10:37:48.661Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2692gitlab # [ 398.573779] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.663Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da3f512b605014d47b278da2","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da3f512b605014d47b278da2: start","job_status":"start","queue_duration_s":0.669605,"scheduling_latency_s":0.669637}2693gitlab # [ 398.589658] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.686Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da3f512b605014d47b278da2","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da3f512b605014d47b278da2: done: 0.023218 sec","job_status":"done","queue_duration_s":0.669605,"scheduling_latency_s":0.669637,"redis_calls":13,"redis_duration_s":0.003492,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001081,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002411,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015588,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023218,"completed_at":"2026-09-01T10:37:48.686Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2694gitlab # [ 398.640069] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.689Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe025d750ddf767ab289daeb","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe025d750ddf767ab289daeb: start","job_status":"start","queue_duration_s":0.694818,"scheduling_latency_s":0.694857}2695gitlab # [ 398.655234] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.721Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe025d750ddf767ab289daeb","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe025d750ddf767ab289daeb: done: 0.032378 sec","job_status":"done","queue_duration_s":0.694818,"scheduling_latency_s":0.694857,"redis_calls":13,"redis_duration_s":0.003086,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001059,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002027,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032378,"completed_at":"2026-09-01T10:37:48.721Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2696gitlab # [ 398.705926] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.723Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b268b2cef1dc94d5bc3b3ff","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b268b2cef1dc94d5bc3b3ff: start","job_status":"start","queue_duration_s":0.729022,"scheduling_latency_s":0.729054}2697gitlab # [ 398.720997] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.745Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b268b2cef1dc94d5bc3b3ff","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b268b2cef1dc94d5bc3b3ff: done: 0.022416 sec","job_status":"done","queue_duration_s":0.729022,"scheduling_latency_s":0.729054,"redis_calls":13,"redis_duration_s":0.003144,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002213,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015635,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022416,"completed_at":"2026-09-01T10:37:48.745Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2698gitlab # [ 398.781972] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.747Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4af097d319ea1f69a9a58cfa","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4af097d319ea1f69a9a58cfa: start","job_status":"start","queue_duration_s":0.753286,"scheduling_latency_s":0.753321}2699gitlab # [ 398.782900] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.769Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4af097d319ea1f69a9a58cfa","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4af097d319ea1f69a9a58cfa: done: 0.022118 sec","job_status":"done","queue_duration_s":0.753286,"scheduling_latency_s":0.753321,"redis_calls":13,"redis_duration_s":0.0032949999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001107,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002188,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022118,"completed_at":"2026-09-01T10:37:48.769Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2700gitlab # [ 398.828460] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.771Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"abcdb110727999775401fefc","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-abcdb110727999775401fefc: start","job_status":"start","queue_duration_s":0.777129,"scheduling_latency_s":0.777161}2701gitlab # [ 398.843388] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.794Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"abcdb110727999775401fefc","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-abcdb110727999775401fefc: done: 0.022865 sec","job_status":"done","queue_duration_s":0.777129,"scheduling_latency_s":0.777161,"redis_calls":13,"redis_duration_s":0.00347,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002376,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015923,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022865,"completed_at":"2026-09-01T10:37:48.794Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2702gitlab # [ 398.891379] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.796Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"decd339e0edcc922a227124a","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-decd339e0edcc922a227124a: start","job_status":"start","queue_duration_s":0.801823,"scheduling_latency_s":0.801851}2703gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses2704gitlab # [ 398.906822] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.818Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"decd339e0edcc922a227124a","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-decd339e0edcc922a227124a: done: 0.022632 sec","job_status":"done","queue_duration_s":0.801823,"scheduling_latency_s":0.801851,"redis_calls":13,"redis_duration_s":0.003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001032,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015865,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022632,"completed_at":"2026-09-01T10:37:48.818Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2705gitlab # [ 398.955072] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b6e12bfe3e9fa219e7eed94","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b6e12bfe3e9fa219e7eed94: start","job_status":"start","queue_duration_s":0.826407,"scheduling_latency_s":0.826441}2706gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses, in 0.09 seconds)2707[{'id': 5, 'sha': 'd996dce18654c3c835478cedbe8d54682367b5ea', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-01T10:37:41.557Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2708gitlab # [ 398.970172] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.840Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b6e12bfe3e9fa219e7eed94","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b6e12bfe3e9fa219e7eed94: done: 0.020421 sec","job_status":"done","queue_duration_s":0.826407,"scheduling_latency_s":0.826441,"redis_calls":13,"redis_duration_s":0.003021,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002113,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012754,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020421,"completed_at":"2026-09-01T10:37:48.840Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2709gitlab # [ 399.019384] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.842Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03637ec54c6bbb3592f58cc6","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03637ec54c6bbb3592f58cc6: start","job_status":"start","queue_duration_s":0.84851,"scheduling_latency_s":0.848539}2710gitlab # [ 399.034960] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.862Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03637ec54c6bbb3592f58cc6","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03637ec54c6bbb3592f58cc6: done: 0.019456 sec","job_status":"done","queue_duration_s":0.84851,"scheduling_latency_s":0.848539,"redis_calls":13,"redis_duration_s":0.003125,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001141,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001984,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019456,"completed_at":"2026-09-01T10:37:48.862Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2711gitlab # [ 399.114188] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.863Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"215bfc022ef05b413f990cd8","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-215bfc022ef05b413f990cd8: start","job_status":"start","queue_duration_s":0.869609,"scheduling_latency_s":0.869635}2712gitlab # [ 399.116306] gitaly[916]: time="2026-09-01T10:37:50.202Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QW6C48DAZRPC089A9CFB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:20.000" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:50.201" grpc.time_ms=0.768 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2713gitlab # [ 399.117407] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:50 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 622714gitlab # [ 399.118051] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"215bfc022ef05b413f990cd8","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-215bfc022ef05b413f990cd8: done: 0.026714 sec","job_status":"done","queue_duration_s":0.869609,"scheduling_latency_s":0.869635,"redis_calls":13,"redis_duration_s":0.010864,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001268,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.009596,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01291,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026714,"completed_at":"2026-09-01T10:37:48.890Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2715gitlab # [ 399.167945] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.892Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f177fd6215136233d173601","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f177fd6215136233d173601: start","job_status":"start","queue_duration_s":0.897828,"scheduling_latency_s":0.897852}2716gitlab # [ 399.227567] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.910Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f177fd6215136233d173601","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f177fd6215136233d173601: done: 0.018802 sec","job_status":"done","queue_duration_s":0.897828,"scheduling_latency_s":0.897852,"redis_calls":13,"redis_duration_s":0.003181,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000953,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002228,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018802,"completed_at":"2026-09-01T10:37:48.910Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2717gitlab # [ 399.232851] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.912Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6aed954a7ecdf806a50e52d9","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6aed954a7ecdf806a50e52d9: start","job_status":"start","queue_duration_s":0.918632,"scheduling_latency_s":0.918662}2718gitlab # [ 399.252173] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.932Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6aed954a7ecdf806a50e52d9","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6aed954a7ecdf806a50e52d9: done: 0.019818 sec","job_status":"done","queue_duration_s":0.918632,"scheduling_latency_s":0.918662,"redis_calls":13,"redis_duration_s":0.003149,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000963,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002186,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012784,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019818,"completed_at":"2026-09-01T10:37:48.932Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2719gitlab # [ 399.346249] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6504d0446ddf73e902272c39","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6504d0446ddf73e902272c39: start","job_status":"start","queue_duration_s":0.939909,"scheduling_latency_s":0.939934}2720gitlab # [ 399.347125] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.954Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6504d0446ddf73e902272c39","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6504d0446ddf73e902272c39: done: 0.020595 sec","job_status":"done","queue_duration_s":0.939909,"scheduling_latency_s":0.939934,"redis_calls":13,"redis_duration_s":0.005374,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000845,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004529,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020595,"completed_at":"2026-09-01T10:37:48.954Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2721gitlab # [ 399.402077] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.956Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4244f7e75fff253a51d795cf","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4244f7e75fff253a51d795cf: start","job_status":"start","queue_duration_s":0.962055,"scheduling_latency_s":0.962081}2722gitlab # [ 399.403217] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.973Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4244f7e75fff253a51d795cf","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4244f7e75fff253a51d795cf: done: 0.017631 sec","job_status":"done","queue_duration_s":0.962055,"scheduling_latency_s":0.962081,"redis_calls":13,"redis_duration_s":0.0029960000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002063,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011432,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017631,"completed_at":"2026-09-01T10:37:48.973Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2723gitlab # [ 399.438258] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.975Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20f1fcde8885127911299c9f","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20f1fcde8885127911299c9f: start","job_status":"start","queue_duration_s":0.981751,"scheduling_latency_s":0.981779}2724gitlab # [ 399.454146] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20f1fcde8885127911299c9f","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20f1fcde8885127911299c9f: done: 0.020285 sec","job_status":"done","queue_duration_s":0.981751,"scheduling_latency_s":0.981779,"redis_calls":13,"redis_duration_s":0.003347,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000784,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002563,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020285,"completed_at":"2026-09-01T10:37:48.996Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2725gitlab # [ 399.555046] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:48.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b21931166b82d14ae169aaf","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b21931166b82d14ae169aaf: start","job_status":"start","queue_duration_s":1.003538,"scheduling_latency_s":1.003561}2726gitlab # [ 399.556146] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.014Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b21931166b82d14ae169aaf","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b21931166b82d14ae169aaf: done: 0.016733 sec","job_status":"done","queue_duration_s":1.003538,"scheduling_latency_s":1.003561,"redis_calls":13,"redis_duration_s":0.0028480000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000847,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016733,"completed_at":"2026-09-01T10:37:49.014Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2727gitlab # [ 399.569997] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.015Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a694741eda98028e2aaf02d3","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a694741eda98028e2aaf02d3: start","job_status":"start","queue_duration_s":1.021744,"scheduling_latency_s":1.021766}2728gitlab # [ 399.586422] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.032Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a694741eda98028e2aaf02d3","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a694741eda98028e2aaf02d3: done: 0.016659 sec","job_status":"done","queue_duration_s":1.021744,"scheduling_latency_s":1.021766,"redis_calls":13,"redis_duration_s":0.002732,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000875,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001857,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016659,"completed_at":"2026-09-01T10:37:49.032Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2729gitlab # [ 399.636804] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.034Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"82fe2d4f07972811569f214b","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-82fe2d4f07972811569f214b: start","job_status":"start","queue_duration_s":1.040129,"scheduling_latency_s":1.040151}2730gitlab # [ 399.652593] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.049Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"82fe2d4f07972811569f214b","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-82fe2d4f07972811569f214b: done: 0.015509 sec","job_status":"done","queue_duration_s":1.040129,"scheduling_latency_s":1.040151,"redis_calls":13,"redis_duration_s":0.002577,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000787,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00179,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015509,"completed_at":"2026-09-01T10:37:49.049Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2731gitlab # [ 399.708282] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.051Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e135114c760db575af26df61","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e135114c760db575af26df61: start","job_status":"start","queue_duration_s":1.056832,"scheduling_latency_s":1.056851}2732gitlab # [ 399.724595] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.065Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e135114c760db575af26df61","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e135114c760db575af26df61: done: 0.014551 sec","job_status":"done","queue_duration_s":1.056832,"scheduling_latency_s":1.056851,"redis_calls":13,"redis_duration_s":0.002558,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00076,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001798,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009499,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014551,"completed_at":"2026-09-01T10:37:49.065Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2733gitlab # [ 399.779294] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.067Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25d2c34a849840a4724ae172","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25d2c34a849840a4724ae172: start","job_status":"start","queue_duration_s":1.072859,"scheduling_latency_s":1.072881}2734gitlab # [ 399.796071] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25d2c34a849840a4724ae172","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25d2c34a849840a4724ae172: done: 0.016144 sec","job_status":"done","queue_duration_s":1.072859,"scheduling_latency_s":1.072881,"redis_calls":13,"redis_duration_s":0.002716,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000805,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001911,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010369,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016144,"completed_at":"2026-09-01T10:37:49.083Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2735gitlab # [ 399.897416] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.084Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a40a8710708cd87928c047f","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a40a8710708cd87928c047f: start","job_status":"start","queue_duration_s":1.090367,"scheduling_latency_s":1.090388}2736gitlab # [ 399.898441] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.098Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a40a8710708cd87928c047f","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a40a8710708cd87928c047f: done: 0.014384 sec","job_status":"done","queue_duration_s":1.090367,"scheduling_latency_s":1.090388,"redis_calls":13,"redis_duration_s":0.00246,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000748,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001712,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009443,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014384,"completed_at":"2026-09-01T10:37:49.098Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2737gitlab # [ 399.971056] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.100Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f1ebc104e01f859e4e37ad8","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f1ebc104e01f859e4e37ad8: start","job_status":"start","queue_duration_s":1.105969,"scheduling_latency_s":1.105988}2738gitlab # [ 399.971885] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.114Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f1ebc104e01f859e4e37ad8","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f1ebc104e01f859e4e37ad8: done: 0.014066 sec","job_status":"done","queue_duration_s":1.105969,"scheduling_latency_s":1.105988,"redis_calls":13,"redis_duration_s":0.002489,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000764,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001725,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009351,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014066,"completed_at":"2026-09-01T10:37:49.114Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2739gitlab # [ 399.983630] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.115Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11c913a6f4f8622188c69857","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11c913a6f4f8622188c69857: start","job_status":"start","queue_duration_s":1.121288,"scheduling_latency_s":1.121307}2740gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses2741gitlab # [ 399.999636] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.130Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11c913a6f4f8622188c69857","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11c913a6f4f8622188c69857: done: 0.014678 sec","job_status":"done","queue_duration_s":1.121288,"scheduling_latency_s":1.121307,"redis_calls":13,"redis_duration_s":0.002474,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000737,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001737,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009699,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014678,"completed_at":"2026-09-01T10:37:49.130Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2742gitlab # [ 400.049280] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.131Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a17dc5200de207ee05458cc6","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a17dc5200de207ee05458cc6: start","job_status":"start","queue_duration_s":1.13733,"scheduling_latency_s":1.137351}2743gitlab # [ 400.065116] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.148Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a17dc5200de207ee05458cc6","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a17dc5200de207ee05458cc6: done: 0.016797 sec","job_status":"done","queue_duration_s":1.13733,"scheduling_latency_s":1.137351,"redis_calls":13,"redis_duration_s":0.002775,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000889,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001886,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011713,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016797,"completed_at":"2026-09-01T10:37:49.148Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2744gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses, in 0.13 seconds)2745[{'id': 5, 'sha': 'd996dce18654c3c835478cedbe8d54682367b5ea', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-01T10:37:41.557Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2746gitlab # [ 400.122670] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.149Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"42194a15ed432f717b9827db","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-42194a15ed432f717b9827db: start","job_status":"start","queue_duration_s":1.155537,"scheduling_latency_s":1.155559}2747gitlab # [ 400.140279] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"42194a15ed432f717b9827db","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-42194a15ed432f717b9827db: done: 0.01589 sec","job_status":"done","queue_duration_s":1.155537,"scheduling_latency_s":1.155559,"redis_calls":13,"redis_duration_s":0.002766,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000855,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001911,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010731,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01589,"completed_at":"2026-09-01T10:37:49.165Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2748gitlab # [ 400.199205] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7d1a215c027da7fbdb895c6","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7d1a215c027da7fbdb895c6: start","job_status":"start","queue_duration_s":1.172859,"scheduling_latency_s":1.172881}2749gitlab # [ 400.256842] gitaly[916]: time="2026-09-01T10:37:51.321Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QX8RF8R6ZBFWF6Z57TP0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:21.119" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:51.319" grpc.time_ms=1.312 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2750gitlab # [ 400.258875] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:51 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 902751gitlab # [ 400.259967] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.185Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7d1a215c027da7fbdb895c6","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7d1a215c027da7fbdb895c6: done: 0.01891 sec","job_status":"done","queue_duration_s":1.172859,"scheduling_latency_s":1.172881,"redis_calls":13,"redis_duration_s":0.003679,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002719,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01891,"completed_at":"2026-09-01T10:37:49.185Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2752gitlab # [ 400.315872] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.187Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a697c24a53b919e87d0a616","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a697c24a53b919e87d0a616: start","job_status":"start","queue_duration_s":1.193212,"scheduling_latency_s":1.193235}2753gitlab # [ 400.317083] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.203Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a697c24a53b919e87d0a616","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a697c24a53b919e87d0a616: done: 0.016287 sec","job_status":"done","queue_duration_s":1.193212,"scheduling_latency_s":1.193235,"redis_calls":13,"redis_duration_s":0.003082,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002206,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010831,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016287,"completed_at":"2026-09-01T10:37:49.203Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2754gitlab # [ 400.346824] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b352691221d9da4ce8b03a53","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b352691221d9da4ce8b03a53: start","job_status":"start","queue_duration_s":1.210905,"scheduling_latency_s":1.210927}2755gitlab # [ 400.362267] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b352691221d9da4ce8b03a53","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b352691221d9da4ce8b03a53: done: 0.015479 sec","job_status":"done","queue_duration_s":1.210905,"scheduling_latency_s":1.210927,"redis_calls":13,"redis_duration_s":0.0024879999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000776,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001712,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010335,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015479,"completed_at":"2026-09-01T10:37:49.220Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2756gitlab # [ 400.428498] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.222Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d3805ed0b65745e8dadfb49","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d3805ed0b65745e8dadfb49: start","job_status":"start","queue_duration_s":1.227846,"scheduling_latency_s":1.227871}2757gitlab # [ 400.429765] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d3805ed0b65745e8dadfb49","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d3805ed0b65745e8dadfb49: done: 0.018234 sec","job_status":"done","queue_duration_s":1.227846,"scheduling_latency_s":1.227871,"redis_calls":13,"redis_duration_s":0.003065,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000979,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002086,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01232,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018234,"completed_at":"2026-09-01T10:37:49.240Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2758gitlab # [ 400.475334] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.241Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96ea22e50bd75b350e4c3937","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96ea22e50bd75b350e4c3937: start","job_status":"start","queue_duration_s":1.247552,"scheduling_latency_s":1.24758}2759gitlab # [ 400.542760] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.261Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96ea22e50bd75b350e4c3937","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96ea22e50bd75b350e4c3937: done: 0.01996 sec","job_status":"done","queue_duration_s":1.247552,"scheduling_latency_s":1.24758,"redis_calls":13,"redis_duration_s":0.003196,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001128,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002068,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01996,"completed_at":"2026-09-01T10:37:49.261Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2760gitlab # [ 400.544910] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.263Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3ec85a99700f1989877ac89","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3ec85a99700f1989877ac89: start","job_status":"start","queue_duration_s":1.269033,"scheduling_latency_s":1.269064}2761gitlab # [ 400.555172] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3ec85a99700f1989877ac89","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3ec85a99700f1989877ac89: done: 0.025079 sec","job_status":"done","queue_duration_s":1.269033,"scheduling_latency_s":1.269064,"redis_calls":13,"redis_duration_s":0.00379,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00125,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00254,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0172,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025079,"completed_at":"2026-09-01T10:37:49.288Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2762gitlab # [ 400.655758] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.290Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c44f91eedf2ce44a1e1f5a73","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c44f91eedf2ce44a1e1f5a73: start","job_status":"start","queue_duration_s":1.296311,"scheduling_latency_s":1.29635}2763gitlab # [ 400.657067] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.321Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c44f91eedf2ce44a1e1f5a73","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c44f91eedf2ce44a1e1f5a73: done: 0.03066 sec","job_status":"done","queue_duration_s":1.296311,"scheduling_latency_s":1.29635,"redis_calls":13,"redis_duration_s":0.004034,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001474,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00256,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018818,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03066,"completed_at":"2026-09-01T10:37:49.321Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2764gitlab # [ 400.668158] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.323Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15a973f1ebdced05bdc986ca","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15a973f1ebdced05bdc986ca: start","job_status":"start","queue_duration_s":1.329438,"scheduling_latency_s":1.329476}2765gitlab # [ 400.683860] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.344Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15a973f1ebdced05bdc986ca","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15a973f1ebdced05bdc986ca: done: 0.020731 sec","job_status":"done","queue_duration_s":1.329438,"scheduling_latency_s":1.329476,"redis_calls":13,"redis_duration_s":0.0029490000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000896,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002053,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012849,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020731,"completed_at":"2026-09-01T10:37:49.344Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2766gitlab # [ 400.732976] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.345Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48a36c83cbf1595cea8a2841","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48a36c83cbf1595cea8a2841: start","job_status":"start","queue_duration_s":1.351691,"scheduling_latency_s":1.351719}2767gitlab # [ 400.748861] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48a36c83cbf1595cea8a2841","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48a36c83cbf1595cea8a2841: done: 0.019422 sec","job_status":"done","queue_duration_s":1.351691,"scheduling_latency_s":1.351719,"redis_calls":13,"redis_duration_s":0.00294,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002023,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01313,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019422,"completed_at":"2026-09-01T10:37:49.365Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2768gitlab # [ 400.798264] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.366Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85beae59c69473b9e011bc95","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85beae59c69473b9e011bc95: start","job_status":"start","queue_duration_s":1.372673,"scheduling_latency_s":1.372703}2769gitlab # [ 400.814422] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85beae59c69473b9e011bc95","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85beae59c69473b9e011bc95: done: 0.020152 sec","job_status":"done","queue_duration_s":1.372673,"scheduling_latency_s":1.372703,"redis_calls":13,"redis_duration_s":0.0037010000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000982,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002719,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013301,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020152,"completed_at":"2026-09-01T10:37:49.386Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2770gitlab # [ 400.864166] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.388Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"406789d0d2b708eb8e3fb836","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-406789d0d2b708eb8e3fb836: start","job_status":"start","queue_duration_s":1.394509,"scheduling_latency_s":1.394537}2771gitlab # [ 400.879589] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.409Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"406789d0d2b708eb8e3fb836","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-406789d0d2b708eb8e3fb836: done: 0.020396 sec","job_status":"done","queue_duration_s":1.394509,"scheduling_latency_s":1.394537,"redis_calls":13,"redis_duration_s":0.003261,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001039,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002222,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013228,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020396,"completed_at":"2026-09-01T10:37:49.409Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2772gitlab # [ 400.930450] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1de1f2ff463b3332d24ad39","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1de1f2ff463b3332d24ad39: start","job_status":"start","queue_duration_s":1.416508,"scheduling_latency_s":1.416536}2773gitlab # [ 400.945517] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1de1f2ff463b3332d24ad39","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1de1f2ff463b3332d24ad39: done: 0.018523 sec","job_status":"done","queue_duration_s":1.416508,"scheduling_latency_s":1.416536,"redis_calls":13,"redis_duration_s":0.002751,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000897,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001854,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012376,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018523,"completed_at":"2026-09-01T10:37:49.429Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2774gitlab # [ 400.995894] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d83161459d874c2e0693f3d3","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d83161459d874c2e0693f3d3: start","job_status":"start","queue_duration_s":1.436574,"scheduling_latency_s":1.436602}2775gitlab # [ 401.012333] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.447Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d83161459d874c2e0693f3d3","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d83161459d874c2e0693f3d3: done: 0.016521 sec","job_status":"done","queue_duration_s":1.436574,"scheduling_latency_s":1.436602,"redis_calls":13,"redis_duration_s":0.002528,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000759,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001769,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011276,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016521,"completed_at":"2026-09-01T10:37:49.447Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2776gitlab # [ 401.061573] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.448Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8490c91cc8e6823446421cae","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8490c91cc8e6823446421cae: start","job_status":"start","queue_duration_s":1.454533,"scheduling_latency_s":1.454558}2777gitlab # [ 401.115545] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.467Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8490c91cc8e6823446421cae","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8490c91cc8e6823446421cae: done: 0.018893 sec","job_status":"done","queue_duration_s":1.454533,"scheduling_latency_s":1.454558,"redis_calls":13,"redis_duration_s":0.00305,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002122,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018893,"completed_at":"2026-09-01T10:37:49.467Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2778gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses2779gitlab # [ 401.123586] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.469Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9d12351d69447a1c7403f3e","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9d12351d69447a1c7403f3e: start","job_status":"start","queue_duration_s":1.47538,"scheduling_latency_s":1.47542}2780gitlab # [ 401.140368] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.487Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9d12351d69447a1c7403f3e","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9d12351d69447a1c7403f3e: done: 0.018226 sec","job_status":"done","queue_duration_s":1.47538,"scheduling_latency_s":1.47542,"redis_calls":13,"redis_duration_s":0.0030069999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00209,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012047,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018226,"completed_at":"2026-09-01T10:37:49.487Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2781gitlab # [ 401.229213] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.489Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf26e6835182b4931f53cb7f","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf26e6835182b4931f53cb7f: start","job_status":"start","queue_duration_s":1.495038,"scheduling_latency_s":1.495062}2782gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses, in 0.11 seconds)2783[{'id': 5, 'sha': 'd996dce18654c3c835478cedbe8d54682367b5ea', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-01T10:37:41.557Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2784gitlab # [ 401.230378] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf26e6835182b4931f53cb7f","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf26e6835182b4931f53cb7f: done: 0.018407 sec","job_status":"done","queue_duration_s":1.495038,"scheduling_latency_s":1.495062,"redis_calls":13,"redis_duration_s":0.003736,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001167,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002569,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011476,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018407,"completed_at":"2026-09-01T10:37:49.507Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2785gitlab # [ 401.252985] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.509Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34f7fa532e2e7338cc89f2af","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34f7fa532e2e7338cc89f2af: start","job_status":"start","queue_duration_s":1.515164,"scheduling_latency_s":1.515187}2786gitlab # [ 401.269750] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.524Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34f7fa532e2e7338cc89f2af","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34f7fa532e2e7338cc89f2af: done: 0.015703 sec","job_status":"done","queue_duration_s":1.515164,"scheduling_latency_s":1.515187,"redis_calls":13,"redis_duration_s":0.002653,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000793,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00186,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010602,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015703,"completed_at":"2026-09-01T10:37:49.524Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2787gitlab # [ 401.320229] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.526Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d4d4f5cc1d12e1d83918f87","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d4d4f5cc1d12e1d83918f87: start","job_status":"start","queue_duration_s":1.532236,"scheduling_latency_s":1.532255}2788gitlab # [ 401.336059] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.547Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d4d4f5cc1d12e1d83918f87","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d4d4f5cc1d12e1d83918f87: done: 0.020628 sec","job_status":"done","queue_duration_s":1.532236,"scheduling_latency_s":1.532255,"redis_calls":13,"redis_duration_s":0.004002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001059,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002943,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01333,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020628,"completed_at":"2026-09-01T10:37:49.547Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2789gitlab # [ 401.385432] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.549Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a30e9a117591531b28155db4","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a30e9a117591531b28155db4: start","job_status":"start","queue_duration_s":1.555229,"scheduling_latency_s":1.55526}2790gitlab # [ 401.401634] gitaly[916]: time="2026-09-01T10:37:52.434Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QYBV1DTAD82TSG1JY3A5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:22.231" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:52.432" grpc.time_ms=1.407 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2791gitlab # [ 401.419049] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 802792gitlab # [ 401.421771] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.567Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a30e9a117591531b28155db4","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a30e9a117591531b28155db4: done: 0.018005 sec","job_status":"done","queue_duration_s":1.555229,"scheduling_latency_s":1.55526,"redis_calls":13,"redis_duration_s":0.00284,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000938,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001902,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018005,"completed_at":"2026-09-01T10:37:49.567Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2793gitlab # [ 401.473288] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.568Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"491dab7849dff0fe5e837c2a","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-491dab7849dff0fe5e837c2a: start","job_status":"start","queue_duration_s":1.574703,"scheduling_latency_s":1.574728}2794gitlab # [ 401.490961] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.592Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"491dab7849dff0fe5e837c2a","created_at":"2026-09-01T10:37:47.928Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-eac59427d0eb4f5c","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"09d288326d73ce9272b88972688d3996","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:47.994Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-491dab7849dff0fe5e837c2a: done: 0.023149 sec","job_status":"done","queue_duration_s":1.574703,"scheduling_latency_s":1.574728,"redis_calls":13,"redis_duration_s":0.003976,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001284,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002692,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015221,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023149,"completed_at":"2026-09-01T10:37:49.591Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2795gitlab # [ 401.556176] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.594Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"70f3e90b45b659087e47e4fb","created_at":"2026-09-01T10:37:48.187Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-0d2933af1da64fba","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ba7d06fd2eef588b9778c36a5b1b7b60","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:48.194Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-70f3e90b45b659087e47e4fb: start","job_status":"start","queue_duration_s":1.40014,"scheduling_latency_s":1.400177}2796gitlab # [ 401.560914] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.610Z","message":"No existing merge request to be cleaned up."}2797gitlab # [ 401.563081] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.620Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"70f3e90b45b659087e47e4fb","created_at":"2026-09-01T10:37:48.187Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-0d2933af1da64fba","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ba7d06fd2eef588b9778c36a5b1b7b60","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:48.194Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-70f3e90b45b659087e47e4fb: done: 0.026101 sec","job_status":"done","queue_duration_s":1.40014,"scheduling_latency_s":1.400177,"redis_calls":10,"redis_duration_s":0.003268,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001259,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002009,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01684,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026101,"completed_at":"2026-09-01T10:37:49.620Z","load_balancing_strategy":"primary","db_duration_s":0.002586,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2798gitlab # [ 401.613066] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.622Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6a1815a8085a3eca8ea72d25","created_at":"2026-09-01T10:37:48.187Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-0d2933af1da64fba","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ba7d06fd2eef588b9778c36a5b1b7b60","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:48.194Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6a1815a8085a3eca8ea72d25: start","job_status":"start","queue_duration_s":1.42826,"scheduling_latency_s":1.428293}2799gitlab # [ 401.631840] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.636Z","message":"No existing merge request to be cleaned up."}2800gitlab # [ 401.633958] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.648Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6a1815a8085a3eca8ea72d25","created_at":"2026-09-01T10:37:48.187Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-0d2933af1da64fba","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ba7d06fd2eef588b9778c36a5b1b7b60","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:48.194Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6a1815a8085a3eca8ea72d25: done: 0.026033 sec","job_status":"done","queue_duration_s":1.42826,"scheduling_latency_s":1.428293,"redis_calls":10,"redis_duration_s":0.003097,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001173,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001924,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016868,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026033,"completed_at":"2026-09-01T10:37:49.648Z","load_balancing_strategy":"primary","db_duration_s":0.002923,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2801gitlab # [ 401.683712] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.650Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a9dfe05585d9273089ee1fdb","created_at":"2026-09-01T10:37:48.187Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-0d2933af1da64fba","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ba7d06fd2eef588b9778c36a5b1b7b60","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:48.194Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a9dfe05585d9273089ee1fdb: start","job_status":"start","queue_duration_s":1.456023,"scheduling_latency_s":1.456052}2802gitlab # [ 401.699100] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.662Z","message":"No existing merge request to be cleaned up."}2803gitlab # [ 401.701241] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.669Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a9dfe05585d9273089ee1fdb","created_at":"2026-09-01T10:37:48.187Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-0d2933af1da64fba","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ba7d06fd2eef588b9778c36a5b1b7b60","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:48.194Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a9dfe05585d9273089ee1fdb: done: 0.019044 sec","job_status":"done","queue_duration_s":1.456023,"scheduling_latency_s":1.456052,"redis_calls":10,"redis_duration_s":0.00244,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001004,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001436,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012387,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019044,"completed_at":"2026-09-01T10:37:49.669Z","load_balancing_strategy":"primary","db_duration_s":0.001807,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2804gitlab # [ 401.764140] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.670Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"979a2a885ac56d339ccf8a61","created_at":"2026-09-01T10:37:48.187Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-0d2933af1da64fba","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ba7d06fd2eef588b9778c36a5b1b7b60","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:48.194Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-979a2a885ac56d339ccf8a61: start","job_status":"start","queue_duration_s":1.476435,"scheduling_latency_s":1.476457}2805gitlab # [ 401.765228] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.680Z","message":"No existing merge request to be cleaned up."}2806gitlab # [ 401.767154] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:49.686Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"979a2a885ac56d339ccf8a61","created_at":"2026-09-01T10:37:48.187Z","trace_propagation_headers":{"sentry-trace":"899fa07b23834929acc9377059b24cbc-0d2933af1da64fba","baggage":"sentry-trace_id=899fa07b23834929acc9377059b24cbc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ba7d06fd2eef588b9778c36a5b1b7b60","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:37:48.194Z","job_size_bytes":2,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-979a2a885ac56d339ccf8a61: done: 0.016486 sec","job_status":"done","queue_duration_s":1.476435,"scheduling_latency_s":1.476457,"redis_calls":10,"redis_duration_s":0.0021349999999999997,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000899,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001236,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016486,"completed_at":"2026-09-01T10:37:49.686Z","load_balancing_strategy":"primary","db_duration_s":0.001775,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2807gitlab # [ 401.973988] gitaly[916]: time="2026-09-01T10:37:53.131Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QZ0RKHCJF62R8Q107WTP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:23.030" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:53.130" grpc.time_ms=0.747 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2808gitlab # [ 402.041620] gitaly[916]: time="2026-09-01T10:37:53.199Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8QZ0RKHCJF62R8Q107WTP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:38:03.121" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:53.191" grpc.time_ms=7.832 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2809gitlab # [ 402.106571] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:53.263Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"518b949f020da086afac41d3","created_at":"2026-09-01T10:37:53.256Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-f4dd1205d68f4de1","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.261Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-518b949f020da086afac41d3: start","job_status":"start","queue_duration_s":0.001902,"scheduling_latency_s":0.001985}2810gitlab # [ 402.148473] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:53 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 201 676 "" "python-httpx/0.28.1" 2572811gitlab # [ 402.152576] nixbot[3225]: {"ts": "2026-09-01T10:37:53.307Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/d996dce18654c3c835478cedbe8d54682367b5ea \"HTTP/1.1 201 Created\""}2812gitlab # [ 402.227427] gitaly[916]: time="2026-09-01T10:37:53.385Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QZ914FRTHQWVWA38PKS0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:23.183" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:53.384" grpc.time_ms=0.726 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2813gitlab # [ 402.276244] gitaly[916]: time="2026-09-01T10:37:53.434Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1E8QZ914FRTHQWVWA38PKS0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:38:03.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:53.427" grpc.time_ms=6.766 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2814gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses2815gitlab # [ 402.363499] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:53 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2072816gitlab # [ 402.366164] nixbot[3225]: {"ts": "2026-09-01T10:37:53.523Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/d996dce18654c3c835478cedbe8d54682367b5ea \"HTTP/1.1 201 Created\""}2817gitlab # [ 402.400105] gitaly[916]: time="2026-09-01T10:37:53.558Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8QZ0RKHCJF62R8Q107WTP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:37:53.557" grpc.time_ms=0.71 pid=916 span.kind=server system=grpc2818gitlab # [ 402.405472] gitaly[916]: time="2026-09-01T10:37:53.563Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QZEQMW11SYSX5A7PA4A7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:23.361" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:53.562" grpc.time_ms=0.695 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2819gitlab # [ 402.443944] gitaly[916]: time="2026-09-01T10:37:53.602Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QZG7CQDR8SKFXFSY4WGW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:23.400" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:53.600" grpc.time_ms=1.185 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2820gitlab # [ 402.466718] gitaly[916]: time="2026-09-01T10:37:53.624Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8QZ0RKHCJF62R8Q107WTP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:37:53.624" grpc.time_ms=0.531 pid=916 span.kind=server system=grpc2821gitlab # [ 402.470173] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:53 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses HTTP/1.1" 200 1314 "" "curl/8.21.0" 1292822gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses, in 0.18 seconds)2823[{'id': 5, 'sha': 'd996dce18654c3c835478cedbe8d54682367b5ea', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 11s (1 warning)', 'created_at': '2026-09-01T10:37:41.557Z', 'started_at': None, 'finished_at': '2026-09-01T10:37:53.243Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'd996dce18654c3c835478cedbe8d54682367b5ea', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-09-01T10:37:53.486Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2824gitlab # [ 402.512260] gitaly[916]: time="2026-09-01T10:37:53.670Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8QZG7CQDR8SKFXFSY4WGW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:38:03.072" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:53.663" grpc.time_ms=6.42 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2825gitlab # [ 402.530661] gitaly[916]: time="2026-09-01T10:37:53.688Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8QZ0RKHCJF62R8Q107WTP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:37:53.687" grpc.time_ms=0.743 pid=916 span.kind=server system=grpc2826gitlab # [ 402.547338] gitaly[916]: time="2026-09-01T10:37:53.705Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QZ0RKHCJF62R8Q107WTP forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T10:38:23.508" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:53.510" operations="[repack_refs]" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2827gitlab # [ 402.561640] gitaly[916]: time="2026-09-01T10:37:53.707Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QZ0RKHCJF62R8Q107WTP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T10:38:23.508" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:53.510" pid=916 remote_ip="2001:db8:1::1" repository_info="{false {6 702 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-09-01 10:36:12.389926199 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root2828gitlab # [ 402.562419] gitaly[916]: time="2026-09-01T10:37:53.709Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QZ0RKHCJF62R8Q107WTP diskcache=fd69498c-d359-4fc3-adbe-51b085c8477f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T10:38:23.508" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:53.510" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2829gitlab # [ 402.563101] gitaly[916]: time="2026-09-01T10:37:53.710Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=106 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7578 command.oublock=0 command.real_time_ms=193 command.spawn_token_fork_ms=0 command.system_time_ms=52 command.user_time_ms=54 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QZ0RKHCJF62R8Q107WTP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T10:38:23.508" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:53.510" grpc.time_ms=199.31 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2830gitlab # [ 402.572170] gitaly[916]: time="2026-09-01T10:37:53.730Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QZ0RKHCJF62R8Q107WTP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T10:38:23.508" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:53.510" pid=916 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2831gitlab # [ 402.573230] gitaly[916]: time="2026-09-01T10:37:53.731Z" level=info msg="optimized repository" clean_stale_data_time=20.925564ms clean_work_trees_time="187.734µs" commit_graph_time="9.219µs" correlation_id=01M1E8QZ0RKHCJF62R8Q107WTP gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="46.934µs" pid=916 prune_time="8.381µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="44.698µs" storage_name=default system=housekeeping total_time=21.513348ms2832gitlab # [ 402.575284] gitaly[916]: time="2026-09-01T10:37:53.731Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QZ0RKHCJF62R8Q107WTP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T10:38:23.508" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:53.510" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2833gitlab # [ 402.644500] gitaly[916]: time="2026-09-01T10:37:53.802Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8QZ0RKHCJF62R8Q107WTP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-01T10:38:23.800" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:53.801" grpc.time_ms=1.012 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2834gitlab # [ 402.657089] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:53 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/d996dce18654c3c835478cedbe8d54682367b5ea HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2712835gitlab # [ 402.658293] nixbot[3225]: {"ts": "2026-09-01T10:37:53.816Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/d996dce18654c3c835478cedbe8d54682367b5ea \"HTTP/1.1 201 Created\""}2836gitlab # [ 402.709066] gitaly[916]: time="2026-09-01T10:37:53.867Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8QZ0RKHCJF62R8Q107WTP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:23.865" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:53.866" grpc.time_ms=0.533 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2837gitlab # [ 402.721588] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:53.878Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"19b18781f7c37337ef4bc733","created_at":"2026-09-01T10:37:53.876Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-192c0f9b6f664163","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"518b949f020da086afac41d3","job_size_bytes":3,"pid":3241,"job_status":"deduplicated","message":"PipelineProcessWorker JID-19b18781f7c37337ef4bc733: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2838gitlab # [ 402.724060] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:53.881Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"518b949f020da086afac41d3","created_at":"2026-09-01T10:37:53.256Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-f4dd1205d68f4de1","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.261Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3241,"job_status":"rescheduled","message":"PipelineProcessWorker JID-518b949f020da086afac41d3: rescheduled"}2839gitlab # [ 402.731489] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:53.889Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"518b949f020da086afac41d3","created_at":"2026-09-01T10:37:53.256Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-f4dd1205d68f4de1","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.261Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-518b949f020da086afac41d3: done: 0.625573 sec","job_status":"done","queue_duration_s":0.001902,"scheduling_latency_s":0.001985,"gitaly_calls":3,"gitaly_duration_s":0.23654,"redis_calls":79,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.023257,"redis_read_bytes":239,"redis_write_bytes":21781,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001561,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001947,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":25,"redis_queues_duration_s":0.007258,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14466,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.007463,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000331,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.0007,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002436,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.08,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.009,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.012,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.039,"db_ci_duration_s":0.041,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.188609,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.446431599,"duration_s":0.625573,"completed_at":"2026-09-01T10:37:53.889Z","load_balancing_strategy":"primary","db_duration_s":0.082239,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2840gitlab # [ 402.773535] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:53.896Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"d254d823ffa817c98bc71b3b","created_at":"2026-09-01T10:37:53.493Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-192c0f9b6f664163","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.496Z","job_size_bytes":23,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-d254d823ffa817c98bc71b3b: start","job_status":"start","queue_duration_s":0.399563,"scheduling_latency_s":0.399609}2841gitlab # [ 402.800614] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:53.957Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"d254d823ffa817c98bc71b3b","created_at":"2026-09-01T10:37:53.493Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-192c0f9b6f664163","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.496Z","job_size_bytes":23,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-d254d823ffa817c98bc71b3b: done: 0.061148 sec","job_status":"done","queue_duration_s":0.399563,"scheduling_latency_s":0.399609,"redis_calls":7,"redis_duration_s":0.004052,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002522,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00153,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0406,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.061148,"completed_at":"2026-09-01T10:37:53.957Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2842gitlab # [ 402.878159] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:53.965Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"bbebc1c7ef37fe3a129537d8","created_at":"2026-09-01T10:37:53.722Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-192c0f9b6f664163","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.726Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-bbebc1c7ef37fe3a129537d8: start","job_status":"start","queue_duration_s":0.238764,"scheduling_latency_s":0.238812}2843gitlab # [ 402.879253] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:54.011Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"bbebc1c7ef37fe3a129537d8","created_at":"2026-09-01T10:37:53.722Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-192c0f9b6f664163","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.726Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-bbebc1c7ef37fe3a129537d8: done: 0.046451 sec","job_status":"done","queue_duration_s":0.238764,"scheduling_latency_s":0.238812,"redis_calls":7,"redis_duration_s":0.0033109999999999997,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002485,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000826,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029672,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.046451,"completed_at":"2026-09-01T10:37:54.011Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005424,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2844gitlab # [ 402.922262] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:54.014Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c44cecca0144186a7be8f05c","created_at":"2026-09-01T10:37:53.736Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-192c0f9b6f664163","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.739Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c44cecca0144186a7be8f05c: start","job_status":"start","queue_duration_s":0.27525,"scheduling_latency_s":0.275309}2845gitlab # [ 402.991466] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:54.051Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c44cecca0144186a7be8f05c","created_at":"2026-09-01T10:37:53.736Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-192c0f9b6f664163","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.739Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c44cecca0144186a7be8f05c: done: 0.03653 sec","job_status":"done","queue_duration_s":0.27525,"scheduling_latency_s":0.275309,"redis_calls":7,"redis_duration_s":0.002151,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00151,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000641,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024975,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03653,"completed_at":"2026-09-01T10:37:54.051Z","load_balancing_strategy":"primary","db_duration_s":0.005784,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2846gitlab # [ 402.993042] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:54.053Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"bb2cca19ac36adff8d45e9e8","created_at":"2026-09-01T10:37:53.751Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-192c0f9b6f664163","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.754Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-bb2cca19ac36adff8d45e9e8: start","job_status":"start","queue_duration_s":0.298956,"scheduling_latency_s":0.298982}2847gitlab # [ 403.011982] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:54.089Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"bb2cca19ac36adff8d45e9e8","created_at":"2026-09-01T10:37:53.751Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-192c0f9b6f664163","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.754Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-bb2cca19ac36adff8d45e9e8: done: 0.036884 sec","job_status":"done","queue_duration_s":0.298956,"scheduling_latency_s":0.298982,"redis_calls":10,"redis_duration_s":0.00264,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001272,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000837,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000531,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02405,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036884,"completed_at":"2026-09-01T10:37:54.089Z","load_balancing_strategy":"primary","db_duration_s":0.006163,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2848gitlab # [ 403.107507] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:54.091Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"96dfc739d58e343c9f0779db","created_at":"2026-09-01T10:37:53.755Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-192c0f9b6f664163","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.757Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-96dfc739d58e343c9f0779db: start","job_status":"start","queue_duration_s":0.333886,"scheduling_latency_s":0.33392}2849gitlab # [ 403.108840] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:54.134Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"96dfc739d58e343c9f0779db","created_at":"2026-09-01T10:37:53.755Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-192c0f9b6f664163","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.757Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-96dfc739d58e343c9f0779db: done: 0.043023 sec","job_status":"done","queue_duration_s":0.333886,"scheduling_latency_s":0.33392,"redis_calls":7,"redis_duration_s":0.001967,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001536,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000431,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026463,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043023,"completed_at":"2026-09-01T10:37:54.134Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008692,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2850gitlab # [ 403.145220] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:54.136Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"d09d0fc751ddfe8847e44eb3","created_at":"2026-09-01T10:37:53.758Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-192c0f9b6f664163","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.762Z","job_size_bytes":19,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-d09d0fc751ddfe8847e44eb3: start","job_status":"start","queue_duration_s":0.374648,"scheduling_latency_s":0.374675}2851gitlab # [ 403.166256] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:54.173Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"d09d0fc751ddfe8847e44eb3","created_at":"2026-09-01T10:37:53.758Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-192c0f9b6f664163","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.762Z","job_size_bytes":19,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-d09d0fc751ddfe8847e44eb3: done: 0.036806 sec","job_status":"done","queue_duration_s":0.374648,"scheduling_latency_s":0.374675,"redis_calls":8,"redis_duration_s":0.002577,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001949,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000628,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025347,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036806,"completed_at":"2026-09-01T10:37:54.173Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005013,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2852gitlab # [ 403.223194] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:54.176Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"9c2e8051882d59b609f97645","created_at":"2026-09-01T10:37:53.764Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-192c0f9b6f664163","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.767Z","job_size_bytes":66,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-9c2e8051882d59b609f97645: start","job_status":"start","queue_duration_s":0.408432,"scheduling_latency_s":0.40846}2853gitlab # [ 403.238926] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:54.213Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"9c2e8051882d59b609f97645","created_at":"2026-09-01T10:37:53.764Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-192c0f9b6f664163","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.767Z","job_size_bytes":66,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-9c2e8051882d59b609f97645: done: 0.037537 sec","job_status":"done","queue_duration_s":0.408432,"scheduling_latency_s":0.40846,"redis_calls":7,"redis_duration_s":0.002937,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002276,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000661,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037537,"completed_at":"2026-09-01T10:37:54.213Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004236,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2854gitlab # [ 403.334112] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:54.216Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"5d1bd1f2cb8a6514a945cab0","created_at":"2026-09-01T10:37:53.768Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-192c0f9b6f664163","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.771Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-5d1bd1f2cb8a6514a945cab0: start","job_status":"start","queue_duration_s":0.444003,"scheduling_latency_s":0.444032}2855gitlab # [ 403.334968] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:54.270Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"5d1bd1f2cb8a6514a945cab0","created_at":"2026-09-01T10:37:53.768Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-192c0f9b6f664163","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.771Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-5d1bd1f2cb8a6514a945cab0: done: 0.054055 sec","job_status":"done","queue_duration_s":0.444003,"scheduling_latency_s":0.444032,"redis_calls":7,"redis_duration_s":0.002332,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001694,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000638,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.054055,"completed_at":"2026-09-01T10:37:54.269Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00812,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2856gitlab # [ 403.363975] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:54.272Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"edae4e07f16ad6fcc87250bb","created_at":"2026-09-01T10:37:53.779Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-192c0f9b6f664163","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.781Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-edae4e07f16ad6fcc87250bb: start","job_status":"start","queue_duration_s":0.490377,"scheduling_latency_s":0.490403}2857gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses2858gitlab # [ 403.383771] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:54.308Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"edae4e07f16ad6fcc87250bb","created_at":"2026-09-01T10:37:53.779Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-192c0f9b6f664163","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.781Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-edae4e07f16ad6fcc87250bb: done: 0.036727 sec","job_status":"done","queue_duration_s":0.490377,"scheduling_latency_s":0.490403,"redis_calls":7,"redis_duration_s":0.002122,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001592,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00053,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022583,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036727,"completed_at":"2026-09-01T10:37:54.308Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007135,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2859gitlab # [ 403.447287] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:54.310Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"013b1735a5741feff366b3bb","created_at":"2026-09-01T10:37:53.782Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-192c0f9b6f664163","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.785Z","job_size_bytes":6,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-013b1735a5741feff366b3bb: start","job_status":"start","queue_duration_s":0.525546,"scheduling_latency_s":0.525593}2860gitlab # [ 403.455681] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:54.332Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"013b1735a5741feff366b3bb","created_at":"2026-09-01T10:37:53.782Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-192c0f9b6f664163","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.785Z","job_size_bytes":6,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-013b1735a5741feff366b3bb: done: 0.021926 sec","job_status":"done","queue_duration_s":0.525546,"scheduling_latency_s":0.525593,"redis_calls":7,"redis_duration_s":0.002582,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001842,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00074,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021926,"completed_at":"2026-09-01T10:37:54.332Z","load_balancing_strategy":"primary","db_duration_s":0.001,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2861gitlab # [ 403.558607] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:54 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1612862gitlab # [ 403.559580] gitlab-workhorse[826]: time="2026-09-01T10:37:54Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672863gitlab # [ 403.560353] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:54 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 642864gitlab # [ 403.561402] gitaly[916]: time="2026-09-01T10:37:54.526Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=24 command.spawn_token_fork_ms=22 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1E8R09YBXPKE4K5X2HD8CDZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:37:54.501" grpc.time_ms=25.468 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc2865gitlab # [ 403.562044] gitaly[916]: time="2026-09-01T10:37:54.592Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R0F1J9G8Q38DCC87VG7A grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:37:54.584" pid=916 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2866gitlab # [ 403.562791] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:54.334Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"886a38ec357ccaa2563da8df","created_at":"2026-09-01T10:37:53.785Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-192c0f9b6f664163","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.787Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-886a38ec357ccaa2563da8df: start","job_status":"start","queue_duration_s":0.547212,"scheduling_latency_s":0.547242}2867gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d996dce18654c3c835478cedbe8d54682367b5ea/statuses, in 0.13 seconds)2868[{'id': 5, 'sha': 'd996dce18654c3c835478cedbe8d54682367b5ea', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 11s (1 warning)', 'created_at': '2026-09-01T10:37:41.557Z', 'started_at': None, 'finished_at': '2026-09-01T10:37:53.243Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'd996dce18654c3c835478cedbe8d54682367b5ea', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-09-01T10:37:53.486Z', 'started_at': None, 'finished_at': '2026-09-01T10:37:53.698Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2869(finished: subtest: eval and build post commit statuses, in 10.43 seconds)2870(finished: run the VM test script, in 466.04 seconds)2871test script finished in 466.08s2872cleanup2873kill QemuMachine (pid 46)2874gitlab # [ 403.563770] sidekiq-cluster[3241]: {"severity":"INFO","time":"2026-09-01T10:37:54.378Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"886a38ec357ccaa2563da8df","created_at":"2026-09-01T10:37:53.785Z","trace_propagation_headers":{"sentry-trace":"4737703629aa4636af461c29ae73065b-192c0f9b6f664163","baggage":"sentry-trace_id=4737703629aa4636af461c29ae73065b,sentry-release=Unknown"},"correlation_id":"01M1E8QZ0RKHCJF62R8Q107WTP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:53.787Z","job_size_bytes":3,"pid":3241,"sidekiq_tid":"m8jl","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-886a38ec357ccaa2563da8df: done: 0.043409 sec","job_status":"done","queue_duration_s":0.547212,"scheduling_latency_s":0.547242,"redis_calls":7,"redis_duration_s":0.003424,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002422,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001002,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duratqemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14)2875gitlab # i2876(finished: cleanup, in 1.91 seconds)