vm-test-run-nixbot-gitlab
checks.x86_64-linux.nixbot-gitlab
· build #296
· raw
1Machine state will be reset. To keep it, pass --keep-machine-state2start all VLans3(finished: start all VLans, in 0.00 seconds)4Test will time out and terminate in 3600 seconds5run the VM test script6additionally exposed symbols:7 gitlab,8 vlan1,9 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh10start all VMs11gitlab: starting vm12gitlab # Disk image does not exist, creating the virtualisation disk image...13gitlab # Formatting '/build/vm-state-gitlab/tmp.EZcbZrxLUU', fmt=raw size=107374182414gitlab # mke2fs 1.47.4 (6-Mar-2025)15gitlab # Discarding device blocks: 0/262144 done16gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes17gitlab # Filesystem UUID: 9f3c290f-e722-4b6c-84b9-c50ee9e9e75318gitlab # Superblock backups stored on blocks:19gitlab # 32768, 98304, 163840, 22937620gitlab # 21gitlab # Allocating group tables: 0/8 done22gitlab # Writing inode tables: 0/8 done23gitlab # Creating journal (8192 blocks): done24gitlab # Writing superblocks and filesystem accounting information: 0/8 done25gitlab # 26gitlab # Virtualisation disk image created.27gitlab # Creating Nix store image...28gitlab # Created Nix store image.29gitlab: QEMU running (pid 46)30(finished: start all VMs, in 56.74 seconds)31gitlab: waiting for unit setup-gitlab.service32gitlab: waiting for the VM to finish booting33gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)34gitlab # 35gitlab # 36gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA0037gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...38gitlab # 39gitlab # 40gitlab # 41gitlab # 42gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB0043gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...44gitlab # 45gitlab # 46gitlab # Booting from ROM...47gitlab # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 202648gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8p04g016fqgdsh7l5fx4rr46qiq28s5q-nixos-system-gitlab-test/init regInfo=/nix/store/f1l04416ni3gvmrzhcqsrc9as5sxlgbr-closure-info/registration console=ttyS0,115200n8 console=tty049gitlab # [ 0.000000] x86 CPU feature dependency check failure: CPU0 has '18*32+31' enabled but '18*32+26' disabled. Kernel might be fine, but no guarantees.50gitlab # [ 0.000000] BIOS-provided physical RAM map:51gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable52gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved53gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved54gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable55gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved56gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved57gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved58gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable59gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved60gitlab # [ 0.000000] NX (Execute Disable) protection: active61gitlab # [ 0.000000] APIC: Static calls initialized62gitlab # [ 0.000000] SMBIOS 2.8 present.63gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/201464gitlab # [ 0.000000] DMI: Memory slots populated: 1/165gitlab # [ 0.000000] Hypervisor detected: KVM66gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x40000000067gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d0068gitlab # [ 0.000002] kvm-clock: using sched offset of 480037285 cycles69gitlab # [ 0.000005] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns70gitlab # [ 0.000011] tsc: Detected 3099.998 MHz processor71gitlab # [ 0.000787] last_pfn = 0x1c0000 max_arch_pfn = 0x40000000072gitlab # [ 0.000813] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs73gitlab # [ 0.000816] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT74gitlab # [ 0.000857] last_pfn = 0xbffd8 max_arch_pfn = 0x40000000075gitlab # [ 0.002940] found SMP MP-table at [mem 0x000f5470-0x000f547f]76gitlab # [ 0.002957] Using GB pages for direct mapping77gitlab # [ 0.003050] RAMDISK: [mem 0xbe31c000-0xbffcffff]78gitlab # [ 0.003056] ACPI: Early table checksum verification disabled79gitlab # [ 0.003060] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )80gitlab # [ 0.003064] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)81gitlab # [ 0.003069] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)82gitlab # [ 0.003075] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001)83gitlab # [ 0.003078] ACPI: FACS 0x00000000BFFE0000 00004084gitlab # [ 0.003079] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001)85gitlab # [ 0.003081] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)86gitlab # [ 0.003083] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)87gitlab # [ 0.003085] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]88gitlab # [ 0.003086] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]89gitlab # [ 0.003087] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]90gitlab # [ 0.003087] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]91gitlab # [ 0.003088] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]92gitlab # [ 0.003088] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]93gitlab # [ 0.003474] No NUMA configuration found94gitlab # [ 0.003476] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]95gitlab # [ 0.003479] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff]96gitlab # [ 0.003578] Zone ranges:97gitlab # [ 0.003579] DMA [mem 0x0000000000001000-0x0000000000ffffff]98gitlab # [ 0.003580] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]99gitlab # [ 0.003582] Normal [mem 0x0000000100000000-0x00000001bfffffff]100gitlab # [ 0.003583] Device empty101gitlab # [ 0.003584] Movable zone start for each node102gitlab # [ 0.003585] Early memory node ranges103gitlab # [ 0.003585] node 0: [mem 0x0000000000001000-0x000000000009efff]104gitlab # [ 0.003586] node 0: [mem 0x0000000000100000-0x00000000bffd7fff]105gitlab # [ 0.003587] node 0: [mem 0x0000000100000000-0x00000001bfffffff]106gitlab # [ 0.003588] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]107gitlab # [ 0.003607] On node 0, zone DMA: 1 pages in unavailable ranges108gitlab # [ 0.003865] On node 0, zone DMA: 97 pages in unavailable ranges109gitlab # [ 0.110722] On node 0, zone Normal: 40 pages in unavailable ranges110gitlab # [ 0.111520] ACPI: PM-Timer IO Port: 0x608111gitlab # [ 0.111535] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])112gitlab # [ 0.111562] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23113gitlab # [ 0.111564] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)114gitlab # [ 0.111566] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)115gitlab # [ 0.111567] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)116gitlab # [ 0.111568] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)117gitlab # [ 0.111569] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)118gitlab # [ 0.111571] ACPI: Using ACPI (MADT) for SMP configuration information119gitlab # [ 0.111572] ACPI: HPET id: 0x8086a201 base: 0xfed00000120gitlab # [ 0.111579] TSC deadline timer available121gitlab # [ 0.111584] CPU topo: Max. logical packages: 1122gitlab # [ 0.111584] CPU topo: Max. logical dies: 1123gitlab # [ 0.111585] CPU topo: Max. dies per package: 1124gitlab # [ 0.111589] CPU topo: Max. threads per core: 1125gitlab # [ 0.111590] CPU topo: Num. cores per package: 4126gitlab # [ 0.111590] CPU topo: Num. threads per package: 4127gitlab # [ 0.111591] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs128gitlab # [ 0.111612] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()129gitlab # [ 0.111640] kvm-guest: KVM setup pv remote TLB flush130gitlab # [ 0.111643] kvm-guest: setup PV sched yield131gitlab # [ 0.111661] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]132gitlab # [ 0.111662] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]133gitlab # [ 0.111663] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]134gitlab # [ 0.111665] [mem 0xc0000000-0xfeffbfff] available for PCI devices135gitlab # [ 0.111666] Booting paravirtualized kernel on KVM136gitlab # [ 0.111670] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns137gitlab # [ 0.116174] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1138gitlab # [ 0.118371] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288139gitlab # [ 0.118416] kvm-guest: PV spinlocks enabled140gitlab # [ 0.118418] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)141gitlab # [ 0.118421] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8p04g016fqgdsh7l5fx4rr46qiq28s5q-nixos-system-gitlab-test/init regInfo=/nix/store/f1l04416ni3gvmrzhcqsrc9as5sxlgbr-closure-info/registration console=ttyS0,115200n8 console=tty0142gitlab # [ 0.118528] Unknown kernel command line parameters "regInfo=/nix/store/f1l04416ni3gvmrzhcqsrc9as5sxlgbr-closure-info/registration", will be passed to user space.143gitlab # [ 0.118546] random: crng init done144gitlab # [ 0.118547] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes145gitlab # [ 0.127302] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)146gitlab # [ 0.131820] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)147gitlab # [ 0.131908] software IO TLB: area num 4.148gitlab # [ 0.210824] Fallback order for Node 0: 0149gitlab # [ 0.210834] Built 1 zonelists, mobility grouping on. Total pages: 1572726150gitlab # [ 0.210836] Policy zone: Normal151gitlab # [ 0.213403] mem auto-init: stack:all(zero), heap alloc:on, heap free:off152gitlab # [ 0.228067] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1153gitlab # [ 0.242666] allocated 12582912 bytes of page_ext154gitlab # [ 0.251908] ftrace: allocating 48717 entries in 192 pages155gitlab # [ 0.251913] ftrace: allocated 192 pages with 2 groups156gitlab # [ 0.252886] Dynamic Preempt: lazy157gitlab # [ 0.253088] rcu: Preemptible hierarchical RCU implementation.158gitlab # [ 0.253089] rcu: RCU event tracing is enabled.159gitlab # [ 0.253090] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.160gitlab # [ 0.253091] Trampoline variant of Tasks RCU enabled.161gitlab # [ 0.253091] Rude variant of Tasks RCU enabled.162gitlab # [ 0.253091] Tracing variant of Tasks RCU enabled.163gitlab # [ 0.253092] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.164gitlab # [ 0.253093] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4165gitlab # [ 0.253116] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.166gitlab # [ 0.253119] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.167gitlab # [ 0.253120] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.168gitlab # [ 0.257915] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16169gitlab # [ 0.258252] rcu: srcu_init: Setting srcu_struct sizes based on contention.170gitlab # [ 0.258262] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns171gitlab # [ 0.258392] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)172gitlab # [ 0.264715] Console: colour VGA+ 80x25173gitlab # [ 0.264723] printk: legacy console [tty0] enabled174gitlab # [ 0.306759] printk: legacy console [ttyS0] enabled175gitlab # [ 0.510005] ACPI: Core revision 20250807176gitlab # [ 0.511328] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns177gitlab # [ 0.513684] APIC: Switch to symmetric I/O mode setup178gitlab # [ 0.515115] x2apic enabled179gitlab # [ 0.516169] APIC: Switched APIC routing to: physical x2apic180gitlab # [ 0.517740] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()181gitlab # [ 0.519853] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()182gitlab # [ 0.522425] kvm-guest: setup PV IPIs183gitlab # [ 0.524781] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1184gitlab # [ 0.526430] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x2caf45e6a57, max_idle_ns: 440795365367 ns185gitlab # [ 0.529199] Calibrating delay loop (skipped) preset value.. 6199.99 BogoMIPS (lpj=3099998)186gitlab # [ 0.531314] x86/cpu: User Mode Instruction Prevention (UMIP) activated187gitlab # [ 0.533193] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127188gitlab # [ 0.534197] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0189gitlab # [ 0.535205] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto190gitlab # [ 0.537197] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl191gitlab # [ 0.539196] Spectre V2 : Mitigation: Retpolines192gitlab # [ 0.541196] RETBleed: Mitigation: untrained return thunk193gitlab # [ 0.542196] Speculative Return Stack Overflow: Mitigation: SMT disabled194gitlab # [ 0.543197] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization195gitlab # [ 0.544197] Spectre V2 : Spectre v2 / SpectreRSB: Filling RSB on context switch and VMEXIT196gitlab # [ 0.545197] Spectre V2 : Enabling Speculation Barrier for firmware calls197gitlab # [ 0.546198] active return thunk: retbleed_return_thunk198gitlab # [ 0.547211] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier199gitlab # [ 0.549213] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'200gitlab # [ 0.551196] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'201gitlab # [ 0.552196] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'202gitlab # [ 0.553197] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256203gitlab # [ 0.554197] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'compacted' format.204gitlab # [ 0.575120] Freeing SMP alternatives memory: 44K205gitlab # [ 0.576198] pid_max: default: 32768 minimum: 301206gitlab # [ 0.577308] LSM: initializing lsm=capability,landlock,yama,bpf,ima207gitlab # [ 0.578295] landlock: Up and running.208gitlab # [ 0.579196] Yama: becoming mindful.209gitlab # [ 0.580376] LSM support for eBPF active210gitlab # [ 0.581449] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)211gitlab # [ 0.582364] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)212gitlab # [ 0.586044] smpboot: CPU0: AMD Ryzen 9 3900 12-Core Processor (family: 0x17, model: 0x71, stepping: 0x0)213gitlab # [ 0.586710] Performance Events: Fam17h+ core perfctr, AMD PMU driver.214gitlab # [ 0.587210] ... version: 0215gitlab # [ 0.588198] ... bit width: 48216gitlab # [ 0.589214] ... generic counters: 6217gitlab # [ 0.590198] ... generic bitmap: 000000000000003f218gitlab # [ 0.591197] ... fixed-purpose counters: 0219gitlab # [ 0.592197] ... fixed-purpose bitmap: 0000000000000000220gitlab # [ 0.593198] ... value mask: 0000ffffffffffff221gitlab # [ 0.594197] ... max period: 00007fffffffffff222gitlab # [ 0.595198] ... global_ctrl mask: 000000000000003f223gitlab # [ 0.596321] signal: max sigframe size: 1776224gitlab # [ 0.597329] rcu: Hierarchical SRCU implementation.225gitlab # [ 0.598202] rcu: Max phase no-delay instances is 400.226gitlab # [ 0.599376] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level227gitlab # [ 0.604335] smp: Bringing up secondary CPUs ...228gitlab # [ 0.605709] smpboot: x86: Booting SMP configuration:229gitlab # [ 0.606203] .... node #0, CPUs: #1 #2 #3230gitlab # [ 0.608278] smp: Brought up 1 node, 4 CPUs231gitlab # [ 0.610199] smpboot: Total of 4 processors activated (24799.98 BogoMIPS)232gitlab # [ 0.612416] Memory: 6013800K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 258088K reserved, 0K cma-reserved)233gitlab # [ 0.614196] devtmpfs: initialized234gitlab # [ 0.615505] x86/mm: Memory block size: 128MB235gitlab # [ 0.618260] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)236gitlab # [ 0.619319] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).237gitlab # [ 0.620348] pinctrl core: initialized pinctrl subsystem238gitlab # [ 0.621563] PM: RTC time: 10:31:36, date: 2026-09-01239gitlab # [ 0.625684] NET: Registered PF_NETLINK/PF_ROUTE protocol family240gitlab # [ 0.627500] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations241gitlab # [ 0.628298] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations242gitlab # [ 0.630299] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations243gitlab # [ 0.631245] audit: initializing netlink subsys (disabled)244gitlab # [ 0.632293] audit: type=2000 audit(1788258696.815:1): state=initialized audit_enabled=0 res=1245gitlab # [ 0.632487] thermal_sys: Registered thermal governor 'fair_share'246gitlab # [ 0.633200] thermal_sys: Registered thermal governor 'bang_bang'247gitlab # [ 0.634198] thermal_sys: Registered thermal governor 'step_wise'248gitlab # [ 0.635197] thermal_sys: Registered thermal governor 'user_space'249gitlab # [ 0.636198] thermal_sys: Registered thermal governor 'power_allocator'250gitlab # [ 0.637307] cpuidle: using governor menu251gitlab # [ 0.640322] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5252gitlab # [ 0.641525] PCI: Using configuration type 1 for base access253gitlab # [ 0.642197] PCI: Using configuration type 1 for extended access254gitlab # [ 0.643362] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.255gitlab # [ 0.645264] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages256gitlab # [ 0.646199] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page257gitlab # [ 0.647199] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages258gitlab # [ 0.648199] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page259gitlab # [ 0.651564] ACPI: Added _OSI(Module Device)260gitlab # [ 0.652199] ACPI: Added _OSI(Processor Device)261gitlab # [ 0.654200] ACPI: Added _OSI(Processor Aggregator Device)262gitlab # [ 0.656404] ACPI: 1 ACPI AML tables successfully acquired and loaded263gitlab # [ 0.661431] ACPI: Interpreter enabled264gitlab # [ 0.662271] ACPI: PM: (supports S0 S3 S4 S5)265gitlab # [ 0.663193] ACPI: Using IOAPIC for interrupt routing266gitlab # [ 0.664338] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug267gitlab # [ 0.666202] PCI: Using E820 reservations for host bridge windows268gitlab # [ 0.668517] ACPI: Enabled 2 GPEs in block 00 to 0F269gitlab # [ 0.672497] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])270gitlab # [ 0.674203] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]271gitlab # [ 0.676696] acpiphp: Slot [3] registered272gitlab # [ 0.678225] acpiphp: Slot [4] registered273gitlab # [ 0.679225] acpiphp: Slot [5] registered274gitlab # [ 0.680246] acpiphp: Slot [6] registered275gitlab # [ 0.681225] acpiphp: Slot [7] registered276gitlab # [ 0.682225] acpiphp: Slot [8] registered277gitlab # [ 0.683225] acpiphp: Slot [9] registered278gitlab # [ 0.684240] acpiphp: Slot [10] registered279gitlab # [ 0.685226] acpiphp: Slot [11] registered280gitlab # [ 0.686225] acpiphp: Slot [12] registered281gitlab # [ 0.687251] acpiphp: Slot [13] registered282gitlab # [ 0.688258] acpiphp: Slot [14] registered283gitlab # [ 0.689229] acpiphp: Slot [15] registered284gitlab # [ 0.690228] acpiphp: Slot [16] registered285gitlab # [ 0.691245] acpiphp: Slot [17] registered286gitlab # [ 0.695242] acpiphp: Slot [18] registered287gitlab # [ 0.696231] acpiphp: Slot [19] registered288gitlab # [ 0.697229] acpiphp: Slot [20] registered289gitlab # [ 0.698256] acpiphp: Slot [21] registered290gitlab # [ 0.699237] acpiphp: Slot [22] registered291gitlab # [ 0.701228] acpiphp: Slot [23] registered292gitlab # [ 0.702223] acpiphp: Slot [24] registered293gitlab # [ 0.703238] acpiphp: Slot [25] registered294gitlab # [ 0.704221] acpiphp: Slot [26] registered295gitlab # [ 0.705222] acpiphp: Slot [27] registered296gitlab # [ 0.706246] acpiphp: Slot [28] registered297gitlab # [ 0.707226] acpiphp: Slot [29] registered298gitlab # [ 0.708223] acpiphp: Slot [30] registered299gitlab # [ 0.709225] acpiphp: Slot [31] registered300gitlab # [ 0.710222] PCI host bridge to bus 0000:00301gitlab # [ 0.711224] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]302gitlab # [ 0.712198] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]303gitlab # [ 0.714198] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]304gitlab # [ 0.716198] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]305gitlab # [ 0.717198] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]306gitlab # [ 0.719199] pci_bus 0000:00: root bus resource [bus 00-ff]307gitlab # [ 0.721542] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint308gitlab # [ 0.724015] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint309gitlab # [ 0.726941] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint310gitlab # [ 0.730263] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f]311gitlab # [ 0.732264] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk312gitlab # [ 0.733203] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk313gitlab # [ 0.735199] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk314gitlab # [ 0.737200] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk315gitlab # [ 0.738768] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint316gitlab # [ 0.743102] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f]317gitlab # [ 0.745181] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint318gitlab # [ 0.747775] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI319gitlab # [ 0.750214] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB320gitlab # [ 0.752682] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint321gitlab # [ 0.763270] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]322gitlab # [ 0.765222] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]323gitlab # [ 0.766247] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]324gitlab # [ 0.768433] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]325gitlab # [ 0.771181] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint326gitlab # [ 0.776225] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]327gitlab # [ 0.778211] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]328gitlab # [ 0.779243] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]329gitlab # [ 0.781214] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]330gitlab # [ 0.785225] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint331gitlab # [ 0.790238] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f]332gitlab # [ 0.797244] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]333gitlab # [ 0.798334] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]334gitlab # [ 0.803337] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint335gitlab # [ 0.809213] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf]336gitlab # [ 0.810210] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]337gitlab # [ 0.811241] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]338gitlab # [ 0.814801] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint339gitlab # [ 0.819226] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df]340gitlab # [ 0.821212] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]341gitlab # [ 0.822249] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]342gitlab # [ 0.825803] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint343gitlab # [ 0.831218] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]344gitlab # [ 0.832212] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]345gitlab # [ 0.834248] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]346gitlab # [ 0.838408] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint347gitlab # [ 0.842189] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff]348gitlab # [ 0.843211] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]349gitlab # [ 0.844247] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]350gitlab # [ 0.848231] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint351gitlab # [ 0.852245] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff]352gitlab # [ 0.854212] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]353gitlab # [ 0.855252] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]354gitlab # [ 0.857223] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]355gitlab # [ 0.860882] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint356gitlab # [ 0.866235] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]357gitlab # [ 0.868247] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]358gitlab # [ 0.871969] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint359gitlab # [ 0.876219] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f]360gitlab # [ 0.878211] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]361gitlab # [ 0.880254] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]362gitlab # [ 0.884421] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint363gitlab # [ 0.888213] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f]364gitlab # [ 0.890210] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]365gitlab # [ 0.891249] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]366gitlab # [ 0.899076] ACPI: PCI: Interrupt link LNKA configured for IRQ 10367gitlab # [ 0.901387] ACPI: PCI: Interrupt link LNKB configured for IRQ 10368gitlab # [ 0.902350] ACPI: PCI: Interrupt link LNKC configured for IRQ 11369gitlab # [ 0.904343] ACPI: PCI: Interrupt link LNKD configured for IRQ 11370gitlab # [ 0.906298] ACPI: PCI: Interrupt link LNKS configured for IRQ 9371gitlab # [ 0.909293] iommu: Default domain type: Translated372gitlab # [ 0.910200] iommu: DMA domain TLB invalidation policy: lazy mode373gitlab # [ 0.912466] ACPI: bus type USB registered374gitlab # [ 0.913274] usbcore: registered new interface driver usbfs375gitlab # [ 0.914325] usbcore: registered new interface driver hub376gitlab # [ 0.916223] usbcore: registered new device driver usb377gitlab # [ 0.918375] NetLabel: Initializing378gitlab # [ 0.919197] NetLabel: domain hash size = 128379gitlab # [ 0.920197] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO380gitlab # [ 0.922242] NetLabel: unlabeled traffic allowed by default381gitlab # [ 0.923209] PCI: Using ACPI for IRQ routing382gitlab # [ 0.925333] pci 0000:00:02.0: vgaarb: setting as boot VGA device383gitlab # [ 0.926193] pci 0000:00:02.0: vgaarb: bridge control possible384gitlab # [ 0.926193] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none385gitlab # [ 0.930200] vgaarb: loaded386gitlab # [ 0.931169] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0387gitlab # [ 0.932134] hpet0: 3 comparators, 64-bit 100.000000 MHz counter388gitlab # [ 0.937378] clocksource: Switched to clocksource kvm-clock389gitlab # [ 0.940720] VFS: Disk quotas dquot_6.6.0390gitlab # [ 0.941929] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)391gitlab # [ 0.943958] pnp: PnP ACPI init392gitlab # [ 0.945363] pnp: PnP ACPI: found 6 devices393gitlab # [ 0.953874] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns394gitlab # [ 0.956492] clocksource: Switched to clocksource acpi_pm395gitlab # [ 0.958204] NET: Registered PF_INET protocol family396gitlab # [ 0.960796] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)397gitlab # [ 0.978596] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)398gitlab # [ 0.980716] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)399gitlab # [ 0.982698] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)400gitlab # [ 0.986926] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)401gitlab # [ 0.988921] TCP: Hash tables configured (established 65536 bind 65536)402gitlab # [ 0.990826] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)403gitlab # [ 0.992809] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)404gitlab # [ 0.994524] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)405gitlab # [ 0.996777] NET: Registered PF_UNIX/PF_LOCAL protocol family406gitlab # [ 0.998288] NET: Registered PF_XDP protocol family407gitlab # [ 0.999479] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]408gitlab # [ 1.000988] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]409gitlab # [ 1.002483] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]410gitlab # [ 1.004097] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]411gitlab # [ 1.005691] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]412gitlab # [ 1.007582] pci 0000:00:01.0: PIIX3: Enabling Passive Release413gitlab # [ 1.009001] pci 0000:00:00.0: Limiting direct PCI/PCI transfers414gitlab # [ 1.011623] ACPI: \_SB_.LNKD: Enabled at IRQ 11415gitlab # [ 1.014572] PCI: CLS 0 bytes, default 64416gitlab # [ 1.015806] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)417gitlab # [ 1.016205] Trying to unpack rootfs image as initramfs...418gitlab # [ 1.017548] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB)419gitlab # [ 1.021063] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x2caf45e6a57, max_idle_ns: 440795365367 ns420gitlab # [ 1.039504] Initialise system trusted keyrings421gitlab # [ 1.041279] workingset: timestamp_bits=40 max_order=21 bucket_order=0422gitlab # [ 1.056311] Key type asymmetric registered423gitlab # [ 1.057437] Asymmetric key parser 'x509' registered424gitlab # [ 1.058866] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)425gitlab # [ 1.061191] io scheduler mq-deadline registered426gitlab # [ 1.062597] io scheduler kyber registered427gitlab # [ 1.068354] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled428gitlab # [ 1.070812] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A429gitlab # [ 1.075161] Linux agpgart interface v0.103430gitlab # [ 1.076337] ACPI: bus type drm_connector registered431gitlab # [ 1.079460] usbcore: registered new interface driver usbserial_generic432gitlab # [ 1.081189] usbserial: USB Serial support registered for generic433gitlab # [ 1.082802] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled434gitlab # [ 1.085094] drop_monitor: Initializing network drop monitor service435gitlab # [ 1.087038] NET: Registered PF_INET6 protocol family436gitlab # [ 1.089411] Segment Routing with IPv6437gitlab # [ 1.090505] In-situ OAM (IOAM) with IPv6438gitlab # [ 1.093296] IPI shorthand broadcast: enabled439gitlab # [ 1.097974] sched_clock: Marking stable (836028673, 261932687)->(1152909391, -54948031)440gitlab # [ 1.100910] registered taskstats version 1441gitlab # [ 1.102622] Loading compiled-in X.509 certificates442gitlab # [ 1.109795] Demotion targets for Node 0: null443gitlab # [ 1.111334] Key type .fscrypt registered444gitlab # [ 1.112404] Key type fscrypt-provisioning registered445gitlab # [ 1.113884] ima: No TPM chip found, activating TPM-bypass!446gitlab # [ 1.115368] ima: Allocated hash algorithm: sha1447gitlab # [ 1.116615] ima: No architecture policies found448gitlab # [ 1.118194] PM: Magic number: 10:629:526449gitlab # [ 1.119392] faux: hash matches450gitlab # [ 1.121193] RAS: Correctable Errors collector initialized.451gitlab # [ 1.126453] clk: Disabling unused clocks452gitlab # [ 1.127519] PM: genpd: Disabling unused power domains453gitlab # [ 1.333906] Freeing initrd memory: 29392K454gitlab # [ 1.338358] Freeing unused decrypted memory: 2028K455gitlab # [ 1.341336] Freeing unused kernel image (initmem) memory: 3640K456gitlab # [ 1.342893] Write protecting the kernel read-only data: 32768k457gitlab # [ 1.345294] Freeing unused kernel image (text/rodata gap) memory: 1220K458gitlab # [ 1.347373] Freeing unused kernel image (rodata/data gap) memory: 760K459gitlab # [ 1.381769] x86/mm: Checked W+X mappings: passed, no W+X pages found.460gitlab # [ 1.383399] Run /init as init process461gitlab # [ 1.401068] systemd[1]: Inserted module 'autofs4'462gitlab # [ 1.424218] fuse: init (API version 7.45)463gitlab # [ 1.433117] ACPI: \_SB_.LNKC: Enabled at IRQ 10464gitlab # [ 1.440349] ACPI: \_SB_.LNKA: Enabled at IRQ 10465gitlab # [ 1.443795] ACPI: \_SB_.LNKB: Enabled at IRQ 11466gitlab # [ 1.484269] systemd[1]: Successfully made /usr/ read-only.467gitlab # [ 1.821430] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)468gitlab # [ 1.830037] systemd[1]: Detected virtualization kvm.469gitlab # [ 1.831672] systemd[1]: Detected architecture x86-64.470gitlab # [ 1.833111] systemd[1]: Running in initrd.471gitlab # [ 1.834610] systemd[1]: Initializing machine ID from random generator.472gitlab # [ 1.836389] systemd[1]: Hostname set to <gitlab>.473gitlab # [ 1.928266] systemd[1]: bpf-restrict-fs: LSM BPF program attached474gitlab # [ 1.961178] systemd[1]: Queued start job for default target Initrd Default Target.475gitlab # [ 1.975703] systemd[1]: Created slice Slice /system/modprobe.476gitlab # [ 1.977627] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.477gitlab # [ 1.979881] systemd[1]: Expecting device /dev/disk/by-label/nix-store...478gitlab # [ 1.981742] systemd[1]: Expecting device /dev/disk/by-label/nixos...479gitlab # [ 1.983523] systemd[1]: Reached target Path Units.480gitlab # [ 1.984978] systemd[1]: Reached target Slice Units.481gitlab # [ 1.986424] systemd[1]: Reached target Swaps.482gitlab # [ 1.987700] systemd[1]: Reached target Timer Units.483gitlab # [ 1.989348] systemd[1]: Listening on D-Bus System Message Bus Socket.484gitlab # [ 1.991257] systemd[1]: Listening on Journal Socket (/dev/log).485gitlab # [ 1.993120] systemd[1]: Listening on Journal Sockets.486gitlab # [ 1.994584] systemd[1]: Listening on udev Control Socket.487gitlab # [ 1.996124] systemd[1]: Listening on udev Kernel Socket.488gitlab # [ 1.997574] systemd[1]: Reached target Socket Units.489gitlab # [ 2.000287] systemd[1]: Starting Create List of Static Device Nodes...490gitlab # [ 2.007437] systemd[1]: Starting Load Kernel Module 9pnet_virtio...491gitlab # [ 2.010384] systemd[1]: Starting Load Kernel Module configfs...492gitlab # [ 2.013993] systemd[1]: Starting Journal Service...493gitlab # [ 2.016383] systemd[1]: Starting Load Kernel Modules...494gitlab # [ 2.017904] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os495gitlab # [ 2.021293] systemd[1]: Starting Coldplug All udev Devices...496gitlab # [ 2.025511] systemd[1]: Finished Create List of Static Device Nodes.497gitlab # [ 2.030878] systemd[1]: modprobe@configfs.service: Deactivated successfully.498gitlab # [ 2.039146] systemd[1]: Finished Load Kernel Module configfs.499gitlab # [ 2.040322] systemd-journald[92]: Collecting audit messages is disabled.500gitlab # [ 2.042685] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config501gitlab # [ 2.050523] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.502gitlab # [ 2.053339] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...503gitlab # [ 2.055017] netfs: FS-Cache loaded504gitlab # [ 2.059028] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev505gitlab # [ 2.067379] 9pnet: Installing 9P2000 support506gitlab # [ 2.069914] systemd[1]: Finished Load Kernel Modules.507gitlab # [ 2.072729] systemd[1]: Starting Apply Kernel Variables...508gitlab # [ 2.082749] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.509gitlab # [ 2.086003] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.510gitlab # [ 2.088887] systemd[1]: Finished Load Kernel Module 9pnet_virtio.511gitlab # [ 2.092111] systemd[1]: Finished Apply Kernel Variables.512gitlab # [ 2.096079] systemd[1]: Starting Create Static Device Nodes in /dev...513gitlab # [ 2.104392] systemd[1]: Started Journal Service.514gitlab # [ 1.842448] systemd-modules-load[93]: Using 4 probe threads515gitlab # [ 1.849643] systemd-modules-load[93]: Inserted module 'virtio_balloon'516gitlab # [ 1.851475] systemd-modules-load[93]: Inserted module 'dm_mod'517gitlab # [ 1.852994] systemd-modules-load[93]: Inserted module 'virtio_gpu'518gitlab # [ 1.854575] systemd[1]: Finished Create Static Device Nodes in /dev.519gitlab # [ 1.859385] systemd[1]: Reached target Preparation for Local File Systems.520gitlab # [ 1.860906] systemd[1]: Reached target Local File Systems.521gitlab # [ 1.862649] systemd[1]: Starting Create System Files and Directories...522gitlab # [ 1.865228] systemd[1]: Starting Rule-based Manager for Device Events and Files...523gitlab # [ 1.876686] systemd[1]: Finished Create System Files and Directories.524gitlab # [ 1.879083] systemd[1]: Finished Coldplug All udev Devices.525gitlab # [ 1.889089] systemd-udevd[111]: Using default interface naming scheme 'v261'.526gitlab # [ 1.902988] systemd[1]: Started Rule-based Manager for Device Events and Files.527gitlab # [ 1.904630] systemd[1]: Reached target System Initialization.528gitlab # [ 1.905886] systemd[1]: Reached target Basic System.529gitlab # [ 2.292443] virtio_blk virtio4: 4/0/0 default/read/poll queues530gitlab # [ 2.296842] uhci_hcd 0000:00:01.2: UHCI Host Controller531gitlab # [ 2.298081] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1532gitlab # [ 2.299874] uhci_hcd 0000:00:01.2: detected 2 ports533gitlab # [ 2.301277] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140534gitlab # [ 2.303439] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)535gitlab # [ 2.304119] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18536gitlab # [ 2.311081] virtio_blk virtio5: 4/0/0 default/read/poll queues537gitlab # [ 2.311530] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1538gitlab # [ 2.314235] usb usb1: Product: UHCI Host Controller539gitlab # [ 2.316296] SCSI subsystem initialized540gitlab # [ 2.317043] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd541gitlab # [ 2.321417] virtio_blk virtio5: [vdb] 10455016 512-byte logical blocks (5.35 GB/4.98 GiB)542gitlab # [ 2.322694] usb usb1: SerialNumber: 0000:00:01.2543gitlab # [ 2.328323] hub 1-0:1.0: USB hub found544gitlab # [ 2.329553] hub 1-0:1.0: 2 ports detected545gitlab # [ 2.353481] scsi host0: ata_piix546gitlab # [ 2.355572] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12547gitlab # [ 2.357566] scsi host1: ata_piix548gitlab # [ 2.358913] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0549gitlab # [ 2.358999] serio: i8042 KBD port at 0x60,0x64 irq 1550gitlab # [ 2.359406] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0551gitlab # [ 2.359751] serio: i8042 AUX port at 0x60,0x64 irq 12552gitlab # [ 2.124541] (udev-worker)[133]: Network interface NamePolicy= disabled on kernel command line.553gitlab # [ 2.126632] (udev-worker)[121]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.554gitlab # [ 2.129960] (udev-worker)[121]: Network interface NamePolicy= disabled on kernel command line.555gitlab # [ 2.147610] systemd[1]: Found device /dev/disk/by-label/nixos.556gitlab # [ 2.149586] systemd[1]: Reached target Initrd Root Device.557gitlab # [ 2.151074] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...558gitlab # [ 2.154318] systemd[1]: Starting Virtual Console Setup...559gitlab # [ 2.166480] systemd[1]: Found device /dev/disk/by-label/nix-store.560gitlab # [ 2.173058] systemd-vconsole-setup[153]: Configuration of first virtual console was skipped, ignoring remaining ones.561gitlab # [ 2.175695] systemd-fsck[156]: nixos: clean, 12/65536 files, 13019/262144 blocks562gitlab # [ 2.177367] systemd[1]: Finished Virtual Console Setup.563gitlab # [ 2.179297] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.564gitlab # [ 2.600579] usb 1-1: new full-speed USB device number 2 using uhci_hcd565gitlab # [ 2.600797] ata2: found unknown device (class 0)566gitlab # [ 2.604126] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100567gitlab # [ 2.609663] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5568gitlab # [ 2.628672] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0569gitlab # [ 2.657305] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray570gitlab # [ 2.684579] cdrom: Uniform CD-ROM driver Revision: 3.20571gitlab # [ 2.766854] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00572gitlab # [ 2.769301] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10573gitlab # [ 2.771080] usb 1-1: Product: QEMU USB Tablet574gitlab # [ 2.772124] usb 1-1: Manufacturer: QEMU575gitlab # [ 2.772987] usb 1-1: SerialNumber: 28754-0000:00:01.2-1576gitlab # [ 2.789818] hid: raw HID events driver (C) Jiri Kosina577gitlab # [ 2.798633] usbcore: registered new interface driver usbhid578gitlab # [ 2.801037] usbhid: USB HID core driver579gitlab # [ 2.807448] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2580gitlab # [ 2.810216] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0581gitlab # [ 2.763357] systemd[1]: Mounting /sysroot...582gitlab # [ 3.139266] EXT4-fs (vda): mounted filesystem 9f3c290f-e722-4b6c-84b9-c50ee9e9e753 r/w with ordered data mode. Quota mode: none.583gitlab # [ 2.881084] systemd[1]: Mounted /sysroot.584gitlab # [ 2.882349] systemd[1]: Reached target Initrd Root File System.585gitlab # [ 2.883948] systemd[1]: Mounting /sysroot/nix/.ro-store...586gitlab # [ 2.885215] systemd[1]: Mounting /sysroot/nix/.rw-store...587gitlab # [ 2.886563] systemd[1]: Mounting /sysroot/run...588gitlab # [ 2.897134] systemd[1]: Mounting /sysroot/tmp/shared...589gitlab # [ 2.899190] systemd[1]: Mounting /sysroot/tmp/xchg...590gitlab # [ 2.900458] systemd[1]: Starting Mountpoints Configured in the Real Root...591gitlab # [ 2.902086] systemd[1]: Mounted /sysroot/run.592gitlab # [ 2.906826] systemd-sysroot-fstab-check[177]: /sysroot should be mounted in the initrd, will request daemon-reload.593gitlab # [ 2.910075] systemd[1]: Mounted /sysroot/nix/.rw-store.594gitlab # [ 2.913458] systemd[1]: Starting rw-sysroot-nix-store.service...595gitlab # [ 3.177117] 9p: Installing v9fs 9p2000 file system support596gitlab # [ 2.915668] systemd[1]: Reload requested from client PID 177 ('syste[ 3.177679] erofs (device vdb): mounted with root inode @ nid 128.597gitlab # md-sysroot') (unit initrd-parse-etc.service)...598gitlab # [ 2.921649] systemd[1]: Reloading...599gitlab # [ 2.983410] systemd[1]: Reloading finished in 69 ms.600gitlab # [ 3.004520] systemd-sysroot-fstab-check[177]: Requesting initrd-fs.target/start/replace...601gitlab # [ 3.006543] systemd[1]: Mounted /sysroot/nix/.ro-store.602gitlab # [ 3.007867] systemd[1]: Mounted /sysroot/tmp/shared.603gitlab # [ 3.009182] systemd[1]: Mounted /sysroot/tmp/xchg.604gitlab # [ 3.010535] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.605gitlab # [ 3.012323] systemd[1]: Finished rw-sysroot-nix-store.service.606gitlab # [ 3.013607] systemd-sysroot-fstab-check[177]: Requesting swap.target/start/replace...607gitlab # [ 3.015358] systemd[1]: initrd-parse-etc.service: Deactivated successfully.608gitlab # [ 3.016899] systemd[1]: Finished Mountpoints Configured in the Real Root.609gitlab # [ 3.018267] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.610gitlab # [ 3.019881] systemd[1]: Starting rw-sysroot-nix-store.service...611gitlab # [ 3.027119] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.612gitlab # [ 3.028702] systemd[1]: Finished rw-sysroot-nix-store.service.613gitlab # [ 3.763698] systemd[1]: Mounting /sysroot/nix/store...614gitlab # [ 3.793215] systemd[1]: Mounted /sysroot/nix/store.615gitlab # [ 3.794497] systemd[1]: Reached target Initrd File Systems.616gitlab # [ 3.795719] systemd[1]: Starting Find NixOS closure...617gitlab # [ 3.796847] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...618gitlab # [ 3.815932] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.619gitlab # [ 3.817703] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.620gitlab # [ 3.826115] systemd[1]: Finished Find NixOS closure.621gitlab # [ 3.827309] systemd[1]: Reached target Initrd Default Target.622gitlab # [ 3.828686] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...623gitlab # [ 3.851543] systemd[1]: Stopped target Initrd Default Target.624gitlab # [ 3.853145] systemd[1]: Stopped target Basic System.625gitlab # [ 3.854417] systemd[1]: Stopped target Initrd Root Device.626gitlab # [ 3.855784] systemd[1]: Stopped target Path Units.627gitlab # [ 3.857131] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.628gitlab # [ 3.858890] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.629gitlab # [ 3.860668] systemd[1]: Stopped target Slice Units.630gitlab # [ 3.862437] systemd[1]: Stopped target Socket Units.631gitlab # [ 3.864088] systemd[1]: Stopped target System Initialization.632gitlab # [ 3.866173] systemd[1]: Stopped target Swaps.633gitlab # [ 3.867920] systemd[1]: Stopped target Timer Units.634gitlab # [ 3.868950] systemd[1]: dbus.socket: Deactivated successfully.635gitlab # [ 3.870282] systemd[1]: Closed D-Bus System Message Bus Socket.636gitlab # [ 3.872045] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.637gitlab # [ 3.874659] systemd[1]: Stopped Find NixOS closure.638gitlab # [ 3.875795] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio639gitlab # [ 3.877880] systemd[1]: Starting rw-sysroot-nix-store.service...640gitlab # [ 3.879130] systemd[1]: systemd-sysctl.service: Deactivated successfully.641gitlab # [ 3.880446] systemd[1]: Stopped Apply Kernel Variables.642gitlab # [ 3.881671] systemd[1]: systemd-modules-load.service: Deactivated successfully.643gitlab # [ 3.883155] systemd[1]: Stopped Load Kernel Modules.644gitlab # [ 3.884211] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.645gitlab # [ 3.885658] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.646gitlab # [ 3.887160] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.647gitlab # [ 3.888539] systemd[1]: Stopped Create System Files and Directories.648gitlab # [ 3.889783] systemd[1]: Stopped target Local File Systems.649gitlab # [ 3.891066] systemd[1]: Stopped target Preparation for Local File Systems.650gitlab # [ 3.892461] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.651gitlab # [ 3.894288] systemd[1]: Stopped Coldplug All udev Devices.652gitlab # [ 3.895634] systemd[1]: Stopping Rule-based Manager for Device Events and Files...653gitlab # [ 3.897311] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.654gitlab # [ 3.899048] systemd[1]: Stopped Virtual Console Setup.655gitlab # [ 3.900495] systemd[1]: initrd-cleanup.service: Deactivated successfully.656gitlab # [ 3.902195] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.657gitlab # [ 3.903481] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.658gitlab # [ 3.904901] systemd[1]: Finished rw-sysroot-nix-store.service.659gitlab # [ 3.906108] systemd[1]: systemd-udevd.service: Deactivated successfully.660gitlab # [ 3.907420] systemd[1]: Stopped Rule-based Manager for Device Events and Files.661gitlab # [ 3.908946] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.662gitlab # [ 3.910478] systemd[1]: Closed udev Control Socket.663gitlab # [ 3.911578] systemd[1]: Starting Cleanup udev Database...664gitlab # [ 3.912799] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.665gitlab # [ 3.914398] systemd[1]: Stopped Create Static Device Nodes in /dev.666gitlab # [ 3.915748] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.667gitlab # [ 3.917424] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.668gitlab # [ 3.919097] systemd[1]: kmod-static-nodes.service: Deactivated successfully.669gitlab # [ 3.920816] systemd[1]: Stopped Create List of Static Device Nodes.670gitlab # [ 3.922183] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.671gitlab # [ 3.923780] systemd[1]: Finished Cleanup udev Database.672gitlab # [ 3.924961] systemd[1]: Reached target Switch Root.673gitlab # [ 3.927095] systemd[1]: Starting NixOS Activation...674gitlab # [ 3.984072] initrd-nixos-activation-start[228]: booting system configuration /nix/store/8p04g016fqgdsh7l5fx4rr46qiq28s5q-nixos-system-gitlab-test675gitlab # [ 4.002712] initrd-nixos-activation-start[228]: running activation script...676gitlab # [ 4.179613] initrd-nixos-activation-start[253]: setting up /etc...677gitlab # [ 4.262389] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.678gitlab # [ 4.264100] systemd[1]: Finished NixOS Activation.679gitlab # [ 4.265294] systemd[1]: Starting Switch Root...680gitlab # [ 4.284045] systemd[1]: Switching root.681gitlab # [ 4.601705] systemd-journald[92]: Received SIGTERM from PID 1 (systemd).682gitlab # [ 4.668807] NET: Registered PF_VSOCK protocol family683gitlab # [ 5.044966] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)684gitlab # [ 5.050669] systemd[1]: Detected virtualization kvm.685gitlab # [ 5.051804] systemd[1]: Detected architecture x86-64.686gitlab # [ 5.053056] systemd[1]: Detected first boot.687gitlab # [ 5.054395] systemd[1]: Initializing machine ID from random generator.688gitlab # [ 5.150694] systemd[1]: bpf-restrict-fs: LSM BPF program attached689gitlab # [ 5.202556] systemd[1]: Applying preset policy.690gitlab # [ 5.293987] systemd[1]: Populated /etc with preset unit settings.691gitlab # [ 5.419132] systemd[1]: initrd-switch-root.service: Deactivated successfully.692gitlab # [ 5.421748] systemd[1]: Stopped initrd-switch-root.service.693gitlab # [ 5.425312] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.694gitlab # [ 5.428230] systemd[1]: Created slice Slice /system/getty.695gitlab # [ 5.429928] systemd[1]: Created slice GitLab DevOps Platform Slice.696gitlab # [ 5.431654] systemd[1]: Created slice User and Session Slice.697gitlab # [ 5.433011] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.698gitlab # [ 5.434689] systemd[1]: Started Forward Password Requests to Wall Directory Watch.699gitlab # [ 5.436197] systemd[1]: Expecting device /dev/hvc0...700gitlab # [ 5.437196] systemd[1]: Expecting device /dev/ttyS0...701gitlab # [ 5.438266] systemd[1]: Reached target Local Encrypted Volumes.702gitlab # [ 5.439468] systemd[1]: Stopped target initrd-fs.target.703gitlab # [ 5.440512] systemd[1]: Stopped target initrd-root-fs.target.704gitlab # [ 5.441618] systemd[1]: Stopped target initrd-switch-root.target.705gitlab # [ 5.443137] systemd[1]: Reached target Virtual Machines and Containers.706gitlab # [ 5.444671] systemd[1]: Reached target Path Units.707gitlab # [ 5.445697] systemd[1]: Reached target Remote File Systems.708gitlab # [ 5.446871] systemd[1]: Reached target Slice Units.709gitlab # [ 5.448013] systemd[1]: Reached target Swaps.710gitlab # [ 5.450212] systemd[1]: Listening on Query the User Interactively for a Password.711gitlab # [ 5.452716] systemd[1]: Listening on Process Core Dump Socket.712gitlab # [ 5.454527] systemd[1]: Listening on Credential Encryption/Decryption.713gitlab # [ 5.456658] systemd[1]: Listening on Factory Reset Management.714gitlab # [ 5.457960] systemd[1]: Listening on Hostname Service Socket.715gitlab # [ 5.460810] systemd[1]: Starting Journal Log Access Socket...716gitlab # [ 5.462477] systemd[1]: Listening on Journal Audit Socket.717gitlab # [ 5.464944] systemd[1]: Listening on Console Output Muting Service Socket.718gitlab # [ 5.466476] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.719gitlab # [ 5.468064] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os720gitlab # [ 5.469823] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki721gitlab # [ 5.473149] systemd[1]: Listening on Disk Repartitioning Service Socket.722gitlab # [ 5.474612] systemd[1]: Listening on udev Control Socket.723gitlab # [ 5.475851] systemd[1]: Listening on udev Varlink Socket.724gitlab # [ 5.479191] systemd[1]: Mounting Huge Pages File System...725gitlab # [ 5.482226] systemd[1]: Mounting POSIX Message Queue File System...726gitlab # [ 5.493591] systemd[1]: Mounting Kernel Debug File System...727gitlab # [ 5.497047] systemd[1]: Mounting Kernel Trace File System...728gitlab # [ 5.501297] systemd[1]: Starting Create List of Static Device Nodes...729gitlab # [ 5.502826] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio730gitlab # [ 5.506767] systemd[1]: Starting Load Kernel Module configfs...731gitlab # [ 5.508126] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm732gitlab # [ 5.510148] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore733gitlab # [ 5.512422] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse734gitlab # [ 5.516270] systemd[1]: Mounting FUSE Control File System...735gitlab # [ 5.517601] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67736gitlab # [ 5.533547] systemd[1]: Starting Journal Service...737gitlab # [ 5.536821] systemd[1]: Starting Load Kernel Modules...738gitlab # [ 5.547224] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...739gitlab # [ 5.550562] systemd[1]: Starting Remount Root and Kernel File Systems...740gitlab # [ 5.554056] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os741gitlab # [ 5.558279] systemd[1]: Starting Coldplug All udev Devices...742gitlab # [ 5.562246] systemd-journald[323]: Collecting audit messages is enabled.743gitlab # [ 5.565200] systemd[1]: Listening on Journal Log Access Socket.744gitlab # [ 5.566847] systemd[1]: Mounted Huge Pages File System.745gitlab # [ 5.568351] systemd[1]: Mounted POSIX Message Queue File System.746gitlab # [ 5.570099] systemd[1]: Mounted Kernel Debug File System.747gitlab # [ 5.571691] systemd[1]: Mounted Kernel Trace File System.748gitlab # [ 5.574338] systemd[1]: Finished Create List of Static Device Nodes.749gitlab # [ 5.576845] systemd[1]: modprobe@configfs.service: Deactivated successfully.750gitlab # [ 5.579841] systemd[1]: Finished Load Kernel Module configfs.751gitlab # [ 5.580363] loop: module loaded752gitlab # [ 5.583330] systemd[1]: Mounted FUSE Control File System.753gitlab # [ 5.322876] systemd[1]: Queued start job for default target Multi-User System[ 5.585181] systemd[1]: Started Journal Service.754gitlab # .755gitlab # [ 5.326135] systemd[1]: Unnecessary job was removed for /dev/vdb.756gitlab # [ 5.588942] EXT4-fs (vda): re-mounted 9f3c290f-e722-4b6c-84b9-c50ee9e9e753.757gitlab # [ 5.329902] systemd[1]: systemd-journald.service: Deactivated successfully.758gitlab # [ 5.331791] systemd-modules-load[324]: Using 3 probe threads759gitlab # [ 5.333321] systemd-modules-load[324]: Inserted module 'tls'760gitlab # [ 5.334852] systemd-modules-load[324]: Inserted module 'loop'761gitlab # [ 5.336338] systemd[1]: Finished Load Kernel Modules.762gitlab # [ 5.337742] systemd[1]: Finished Remount Root and Kernel File Systems.763gitlab # [ 5.339541] systemd-oomd[328]: No swap; memory pressure usage will be degraded764gitlab # [ 5.341785] systemd[1]: Listening on Disk Image Download Service Socket.765gitlab # [ 5.343469] systemd[1]: Mounting Kernel Configuration File System...766gitlab # [ 5.345594] systemd[1]: Starting Firewall...767gitlab # [ 5.346693] systemd[1]: Starting Flush Journal to Persistent Storage...768gitlab # [ 5.348142] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore769gitlab # [ 5.350943] systemd[1]: Starting Load/Save OS Random Seed...770gitlab # [ 5.356121] systemd[1]: Starting Apply Kernel Variables...771gitlab # [ 5.359230] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...772gitlab # [ 5.360882] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os773gitlab # [ 5.625697] systemd-journald[323]: Received client request to flush runtime journal.774gitlab # [ 5.435610] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.775gitlab # [ 5.438119] systemd[1]: Mounted Kernel Configuration File System.776gitlab # [ 5.441051] systemd[1]: Finished Load/Save OS Random Seed.777gitlab # [ 5.443884] systemd[1]: Reached target First Boot Complete.778gitlab # [ 5.445548] systemd[1]: Finished Apply Kernel Variables.779gitlab # [ 5.447226] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.780gitlab # [ 5.448948] systemd[1]: Starting Create Static Device Nodes in /dev...781gitlab # [ 5.450406] systemd[1]: Finished Create Static Device Nodes in /dev.782gitlab # [ 5.451669] systemd[1]: Reached target Preparation for Local File Systems.783gitlab # [ 5.453673] systemd[1]: Starting Rule-based Manager for Device Events and Files...784gitlab # [ 5.455458] systemd[1]: Finished Flush Journal to Persistent Storage.785gitlab # [ 5.458427] systemd[1]: Finished Coldplug All udev Devices.786gitlab # [ 5.462955] systemd-udevd[361]: Using default interface naming scheme 'v261'.787gitlab # [ 5.496929] systemd[1]: Started Rule-based Manager for Device Events and Files.788gitlab # [ 5.603967] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse789gitlab # [ 5.616451] systemd[1]: Finished Firewall.790gitlab # [ 5.626803] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.791gitlab # [ 5.636725] (udev-worker)[462]: Network interface NamePolicy= disabled on kernel command line.792gitlab # [ 5.644170] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.793gitlab # [ 5.646781] (udev-worker)[451]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.794gitlab # [ 5.650188] (udev-worker)[451]: Network interface NamePolicy= disabled on kernel command line.795gitlab # [ 5.658870] systemd[1]: Condition check resulted in Virtio network device being skipped.796gitlab # [ 5.660898] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore797gitlab # [ 5.663393] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67798gitlab # [ 5.666912] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore799gitlab # [ 5.669714] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os800gitlab # [ 5.671824] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os801gitlab # [ 5.967044] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3802gitlab # [ 5.985025] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console803gitlab # [ 5.985158] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0804gitlab # [ 5.988023] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD805gitlab # [ 5.988159] mousedev: PS/2 mouse device common for all mice806gitlab # [ 5.991448] ACPI: button: Power Button [PWRF]807gitlab # [ 5.994674] Console: switching to colour dummy device 80x25808gitlab # [ 5.999170] [drm] Found bochs VGA, ID 0xb0c5.809gitlab # [ 5.999174] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.810gitlab # [ 6.002714] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic811gitlab # [ 6.004654] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0812gitlab # [ 6.047331] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5813gitlab # [ 6.047477] Console: switching to colour frame buffer device 160x50814gitlab # [ 6.050679] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6815gitlab # [ 6.053310] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device816gitlab # [ 6.088295] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4817gitlab # [ 6.153395] rtc_cmos 00:05: RTC can wake from S4818gitlab # [ 6.156565] parport_pc 00:03: reported by Plug and Play ACPI819gitlab # [ 6.158526] Floppy drive(s): fd0 is 2.88M AMI BIOS820gitlab # [ 6.160804] rtc_cmos 00:05: registered as rtc0821gitlab # [ 6.160817] parport0: PC-style at 0x378822gitlab # [ 6.161173] rtc_cmos 00:05: setting system clock to 2026-09-01T10:31:41 UTC (1788258701)823gitlab # [ 6.161669] , irq 7 [PCSPP(,...)]824gitlab # [ 6.165108] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs825gitlab # [ 6.173220] FDC 0 is a S82078B826gitlab # [ 6.175463] systemd-journald[323]: Time jumped backwards, rotating.827gitlab # [ 5.942261] systemd[1]: Starting Virtual Console Setup...828gitlab # [ 6.206696] erofs (device vdb): failed to read inode meta block (nid: 109654912): -4829gitlab # [ 5.950081] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.830gitlab # [ 5.953519] s[ 6.215470] kvm_amd: TSC scaling supported831gitlab # ystemd[1]: Stopped Virtual Console Setup.[ 6.216981] kvm_amd: Nested Virtualization enabled832gitlab # [ 6.217750] kvm_amd: Nested Paging enabled833gitlab # 834gitlab # [ 6.218538] kvm_amd: LBR virtualization supported835gitlab # [ 6.219448] kvm_amd: Virtual VMLOAD VMSAVE supported836gitlab # [ 6.220351] kvm_amd: Virtual GIF supported837gitlab # [ 5.961412] systemd[1]: Starting Virtual Console Setup...838gitlab # [ 5.976702] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.839gitlab # [ 5.978257] systemd[1]: Stopped Virtual Console Setup.840gitlab # [ 5.979985] systemd[1]: Starting Virtual Console Setup...841gitlab # [ 6.244145] ppdev: user-space parallel port driver842gitlab # [ 6.265701] EDAC MC: Ver: 3.0.0843gitlab # [ 6.017078] systemd-vconsole-setup[500]: Configuration of first virtual console was skipped, ignoring remaining ones.844gitlab # [ 6.020428] systemd[1]: Finished Virtual Console Setup.845gitlab # [ 6.158673] systemd[1]: Mounting /run/wrappers...846gitlab # [ 6.183001] systemd[1]: Mounted /run/wrappers.847gitlab # [ 6.192600] systemd[1]: Reached target Local File Systems.848gitlab # [ 6.194223] systemd[1]: Listening on Boot Loader Control Service Socket.849gitlab # [ 6.195925] systemd[1]: Starting register-nix-paths.service...850gitlab # [ 6.199208] systemd[1]: Starting Create SUID/SGID Wrappers...851gitlab # [ 6.200616] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.852gitlab # [ 6.205213] systemd[1]: Starting Save Transient machine-id to Disk...853gitlab # [ 6.206646] systemd[1]: Starting Create System Files and Directories...854gitlab # [ 6.229845] systemd[1]: Finished Create System Files and Directories.855gitlab # [ 6.233723] systemd[1]: Starting Rebuild Journal Catalog...856gitlab # [ 6.235802] systemd[1]: Starting Record System Boot/Shutdown in UTMP...857gitlab # [ 6.244210] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.858gitlab # [ 6.246514] systemd[1]: Finished Save Transient machine-id to Disk.859gitlab # [ 6.256188] systemd[1]: Finished Record System Boot/Shutdown in UTMP.860gitlab # [ 6.267265] systemd[1]: Finished Rebuild Journal Catalog.861gitlab # [ 6.269951] systemd[1]: Starting Update is Completed...862gitlab # [ 6.287681] systemd[1]: Finished Update is Completed.863gitlab # [ 6.324810] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.864gitlab # [ 6.326607] systemd[1]: Finished Create SUID/SGID Wrappers.865gitlab # [ 6.565454] systemd[1]: Finished register-nix-paths.service.866gitlab # [ 6.567220] systemd[1]: Reached target System Initialization.867gitlab # [ 6.568562] systemd[1]: Started Discard unused filesystem blocks once a week.868gitlab # [ 6.570046] systemd[1]: Started logrotate.timer.869gitlab # [ 6.571227] systemd[1]: Started Daily Cleanup of Temporary Directories.870gitlab # [ 6.572863] systemd[1]: Reached target Timer Units.871gitlab # [ 6.574080] systemd[1]: Listening on D-Bus System Message Bus Socket.872gitlab # [ 6.575502] systemd[1]: Listening on Nix Daemon Socket.873gitlab # [ 6.576808] systemd[1]: Starting nixbot CI service socket...874gitlab # [ 6.578185] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.875gitlab # [ 6.580112] systemd[1]: Listening on nixbot CI service socket.876gitlab # [ 6.581575] systemd[1]: Reached target Socket Units.877gitlab # [ 6.582988] systemd[1]: Starting D-Bus System Message Bus...878gitlab # [ 6.603070] dbus-broker-launch[582]: Looking up NSS user entry for 'systemd-timesync'...879gitlab # [ 6.605625] dbus-broker-launch[582]: NSS returned no entry for 'systemd-timesync'880gitlab # [ 6.607292] dbus-broker-launch[582]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"881gitlab # [ 6.610270] systemd[1]: Started D-Bus System Message Bus.882gitlab # [ 6.620199] dbus-broker-launch[582]: Ready883gitlab # [ 6.622231] systemd[1]: Reached target Basic System.884gitlab # [ 6.623868] systemd[1]: Started backdoor.service.885gitlab # [ 6.625222] systemd[1]: Starting gitlab-config.service...886gitlab # [ 6.627613] systemd[1]: Starting Import lastlog data into lastlog2 database...887gitlab # [ 6.640297] systemd[1]: Starting Logrotate configuration check...888gitlab # [ 6.645110] systemd[1]: Starting Name Service Cache Daemon (nsncd)...889gitlab # [ 6.649069] systemd[1]: Starting Post-Boot Actions...890gitlab # [ 6.653169] systemd[1]: Started Reset console on configuration changes.891gitlab # [ 6.656123] systemd[1]: Starting resolvconf update...892gitlab # connecting to host...893gitlab # [ 6.659502] logrotate[587]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.894gitlab # [ 6.661359] logrotate[587]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf895gitlab: Guest shell says: b'Spawning backdoor root shell...\n'896gitlab # [ 6.662156] logrotate[587]: note: 'monthly' overrides previously specified 'weekly'897gitlab # [ 6.670188] logrotate[587]: Reading state from file: /var/lib/logrotate.status898gitlab # [ 6.670843] logrotate[587]: state file /var/lib/logrotate.status does not exist899gitlab # [ 6.671667] logrotate[587]: Allocating hash table for state file, size 64 entries900gitlab # [ 6.672712] logrotate[587]: Handling 4 logs901gitlab: connected to guest root shell902gitlab # [ 6.673774] logrotate[587]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed903gitlab: (connecting took 7.48 seconds)904gitlab # [ 6.674834] logrotate[587]: considering log /var/log/btmp905gitlab: (finished: waiting for the VM to finish booting, in 7.48 seconds)906gitlab # [ 6.677574] logrotate[587]: Creating new state907gitlab # [ 6.678194] logrotate[587]: Now: 2026-09-01 10:31908gitlab # [ 6.678831] logrotate[587]: Last rotated at 2026-09-01 10:00909gitlab # [ 6.679778] logrotate[587]: log does not need rotating (log has already been rotated)910gitlab # [ 6.680807] logrotate[587]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed911gitlab # [ 6.681717] logrotate[587]: considering log /var/log/wtmp912gitlab # [ 6.682501] logrotate[587]: Creating new state913gitlab # [ 6.683188] logrotate[587]: Now: 2026-09-01 10:31914gitlab # [ 6.683855] logrotate[587]: Last rotated at 2026-09-01 10:00915gitlab # [ 6.685154] logrotate[587]: log does not need rotating (log has already been rotated)916gitlab # [ 6.685959] logrotate[587]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed917gitlab # [ 6.686898] logrotate[587]: switching euid from 0 to 165 and egid from 0 to 165 (pid 587)918gitlab # [ 6.687977] logrotate[587]: considering log /var/gitlab/state/log/*.log919gitlab # [ 6.688842] logrotate[587]: log /var/gitlab/state/log/*.log does not exist -- skipping920gitlab # [ 6.689679] logrotate[587]: Creating new state921gitlab # [ 6.690700] logrotate[587]: switching euid from 165 to 0 and egid from 165 to 0 (pid 587)922gitlab # [ 6.691939] logrotate[587]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed923gitlab # [ 6.692937] logrotate[587]: switching euid from 0 to 60 and egid from 0 to 60 (pid 587)924gitlab # [ 6.693981] logrotate[587]: considering log /var/log/nginx/*.log925gitlab # [ 6.695070] logrotate[587]: log /var/log/nginx/*.log does not exist -- skipping926gitlab # [ 6.695828] logrotate[587]: Creating new state927gitlab # [ 6.696817] logrotate[587]: not running postrotate script, since no logs were rotated928gitlab # [ 6.697866] logrotate[587]: switching euid from 60 to 0 and egid from 60 to 0 (pid 587)929gitlab # [ 6.699139] nsncd[588]: Sep 01 10:31:42.286 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"930gitlab # [ 6.700106] systemd[1]: Finished Logrotate configuration check.931gitlab # [ 6.701517] systemd[1]: Started Name Service Cache Daemon (nsncd).932gitlab # [ 6.702498] systemd[1]: Finished Post-Boot Actions.933gitlab # [ 6.716751] systemd[1]: Finished Import lastlog data into lastlog2 database.934gitlab # [ 6.726762] systemd[1]: Reached target Host and Network Name Lookups.935gitlab # [ 6.728132] systemd[1]: Reached target User and Group Name Lookups.936gitlab # [ 6.736652] systemd[1]: Starting User Login Management...937gitlab # [ 6.779087] systemd[1]: Stopped target Host and Network Name Lookups.938gitlab # [ 6.779427] systemd[1]: Stopping Host and Network Name Lookups...939gitlab # [ 6.779642] systemd[1]: Stopped target User and Group Name Lookups.940gitlab # [ 6.779875] systemd[1]: Stopping User and Group Name Lookups...941gitlab # [ 6.788931] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...942gitlab # [ 6.791990] systemd[1]: nscd.service: Deactivated successfully.943gitlab # [ 6.794124] systemd[1]: Stopped Name Service Cache Daemon (nsncd).944gitlab # [ 6.794409] systemd[1]: nscd.service: Consumed 23ms CPU time over 147ms wall clock time, 7.4M memory peak, 5.5M read from disk.945gitlab # [ 6.803911] systemd[1]: Starting Name Service Cache Daemon (nsncd)...946gitlab # [ 6.822303] systemd[1]: Started Name Service Cache Daemon (nsncd).947gitlab # [ 6.823616] nsncd[675]: Sep 01 10:31:42.423 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"948gitlab # [ 6.824527] systemd[1]: Reached target Host and Network Name Lookups.949gitlab # [ 6.825349] systemd[1]: Reached target User and Group Name Lookups.950gitlab # [ 6.831755] systemd-logind[644]: New seat seat0.951gitlab # [ 6.833829] systemd-logind[644]: Watching system buttons on /dev/input/event2 (Power Button)952gitlab # [ 6.835310] systemd-logind[644]: Watching system buttons on /dev/input/event4 (QEMU Virtio Keyboard)953gitlab # [ 6.837189] systemd-logind[644]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)954gitlab # [ 6.837892] systemd[1]: Started User Login Management.955gitlab # [ 6.840868] systemd[1]: Starting linger-users.service...956gitlab # [ 6.849132] systemd[1]: Finished resolvconf update.957gitlab # [ 6.850690] systemd[1]: Reached target Preparation for Network.958gitlab # [ 6.854937] systemd[1]: Starting DHCP Client...959gitlab # [ 6.860998] systemd[1]: Starting Address configuration of eth1...960gitlab # [ 6.868290] systemd[1]: Starting Extra networking commands....961gitlab # [ 6.870364] systemd[1]: linger-users.service: Deactivated successfully.962gitlab # [ 6.875056] systemd[1]: Finished linger-users.service.963gitlab # [ 6.889888] network-addresses-eth1-start[704]: adding address 192.168.1.1/24... done964gitlab # [ 6.894809] network-addresses-eth1-start[704]: adding address 2001:db8:1::1/64... done965gitlab # [ 6.898436] systemd[1]: Finished Address configuration of eth1.966gitlab # [ 6.932313] systemd[1]: Finished Extra networking commands..967gitlab # [ 6.935970] systemd[1]: Reached target Network.968gitlab # [ 6.941217] systemd[1]: Starting gitlab-workhorse.service...969gitlab # [ 6.943681] systemd[1]: Starting Nginx Web Server...970gitlab # [ 6.948272] systemd[1]: Starting PostgreSQL Server...971gitlab # [ 6.952064] dhcpcd[730]: dhcpcd-10.3.2 starting972gitlab # [ 6.953655] dhcpcd[773]: dev: loaded udev973gitlab # [ 6.955447] systemd[1]: Starting Redis Server - redis-gitlab...974gitlab # [ 6.958793] systemd[1]: Starting Permit User Sessions...975gitlab # [ 7.224407] 8021q: 802.1Q VLAN Support v1.8976gitlab # [ 7.225689] 8021q: adding VLAN 0 to HW filter on device eth1977gitlab # [ 6.977548] systemd[1]: Finished Permit User Sessions.978gitlab # [ 6.982567] systemd[1]: Started Getty on tty1.979gitlab # [ 6.985442] systemd[1]: Reached target Login Prompts.980gitlab # [ 6.992139] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.981gitlab # [ 7.346531] cfg80211: Loading compiled-in X.509 certificates for regulatory database982gitlab # [ 7.352996] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'983gitlab # [ 7.355623] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'984gitlab # [ 7.358399] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2985gitlab # [ 7.360455] cfg80211: failed to load regulatory.db986gitlab # [ 7.099884] nginx-pre-start[793]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok987gitlab # [ 7.101986] nginx-pre-start[793]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful988gitlab # [ 7.108718] systemd[1]: Started Nginx Web Server.989gitlab # [ 7.123398] redis-server[795]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.990gitlab # [ 7.126510] redis-server[795]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo991gitlab # [ 7.128587] redis-server[795]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=795, just started992gitlab # [ 7.131142] redis-server[795]: Configuration loaded993gitlab # [ 7.133384] systemd[1]: Started Redis Server - redis-gitlab.994gitlab # [ 7.396560] 8021q: adding VLAN 0 to HW filter on device eth0995gitlab # [ 7.136271] postgresql-pre-start[797]: The files belonging to this database system will be owned by user "postgres".996gitlab # [ 7.138980] postgresql-pre-start[797]: This user must also own the server process.997gitlab # [ 7.143324] redis[795]: monotonic clock: POSIX clock_gettime998gitlab # [ 7.145123] postgresql-pre-start[797]: The database cluster will be initialized with locale "en_US.UTF-8".999gitlab # [ 7.147177] postgresql-pre-start[797]: The default database encoding has accordingly been set to "UTF8".1000gitlab # [ 7.148930] postgresql-pre-start[797]: The default text search configuration will be set to "english".1001gitlab # [ 7.150742] postgresql-pre-start[797]: Data page checksums are enabled.1002gitlab # [ 7.152465] postgresql-pre-start[797]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok1003gitlab # [ 7.154813] postgresql-pre-start[797]: creating subdirectories ... ok1004gitlab # [ 7.156445] postgresql-pre-start[797]: selecting dynamic shared memory implementation ... posix1005gitlab # [ 7.158431] redis[795]: Running mode=standalone, port=0.1006gitlab # [ 7.160198] redis[795]: Server initialized1007gitlab # [ 7.161622] redis[795]: Ready to accept connections unix1008gitlab # [ 7.163346] redis[795]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1009gitlab # [ 7.166521] dhcpcd[773]: eth0: waiting for carrier1010gitlab # [ 7.167879] postgresql-pre-start[797]: selecting default "max_connections" ... 1001011gitlab # [ 7.169666] dhcpcd[773]: eth0: carrier acquired1012gitlab # [ 7.170891] dhcpcd[773]: DUID 00:01:00:01:32:29:66:0e:52:54:00:12:34:561013gitlab # [ 7.172500] dhcpcd[773]: eth0: IAID 00:12:34:561014gitlab # [ 7.173580] dhcpcd[773]: eth0: adding address fe80::5054:ff:fe12:34561015gitlab # [ 7.187211] dhcpcd[773]: eth0: soliciting a DHCP lease1016gitlab # [ 7.454165] NET: Registered PF_PACKET protocol family1017gitlab # [ 7.193399] postgresql-pre-start[797]: selecting default "shared_buffers" ... 128MB1018gitlab # [ 7.199936] dhcpcd[773]: eth0: offered 10.0.2.15 from 10.0.2.21019gitlab # [ 7.205771] dhcpcd[773]: eth0: probing address 10.0.2.15/241020gitlab # [ 7.440628] postgresql-pre-start[797]: selecting default time zone ... UTC1021gitlab # [ 7.443460] postgresql-pre-start[797]: creating configuration files ... ok1022gitlab # [ 7.477508] systemd[1]: Started gitlab-workhorse.service.1023gitlab # [ 7.620142] postgresql-pre-start[797]: running bootstrap script ... ok1024gitlab # [ 7.694541] gitlab-workhorse[832]: time="2026-09-01T10:31:43Z" level=info msg=Starting build_time=19700101.000000 version=19.0.41025gitlab # [ 7.696978] gitlab-workhorse[832]: time="2026-09-01T10:31:43Z" level=info msg="Using redis/go-redis"1026gitlab # [ 7.702271] gitlab-workhorse[832]: time="2026-09-01T10:31:43Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1027gitlab # [ 8.101929] postgresql-pre-start[797]: performing post-bootstrap initialization ... ok1028gitlab # [ 8.309569] postgresql-pre-start[797]: syncing data to disk ... ok1029gitlab # [ 8.310294] postgresql-pre-start[797]: initdb: warning: enabling "trust" authentication for local connections1030gitlab # [ 8.311066] postgresql-pre-start[797]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1031gitlab # [ 8.311888] postgresql-pre-start[797]: Success. You can now start the database server using:1032gitlab # [ 8.312987] postgresql-pre-start[797]: pg_ctl -D /var/lib/postgresql/18 -l logfile start1033gitlab # [ 8.455647] postgres[856]: [856] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1034gitlab # [ 8.458351] postgres[856]: [856] LOG: listening on IPv6 address "::1", port 54321035gitlab # [ 8.460371] postgres[856]: [856] LOG: listening on IPv4 address "127.0.0.1", port 54321036gitlab # [ 8.462331] postgres[856]: [856] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1037gitlab # [ 8.468892] postgres[866]: [866] LOG: database system was shut down at 2026-09-01 10:31:43 GMT1038gitlab # [ 8.476141] postgres[856]: [856] LOG: database system is ready to accept connections1039gitlab # [ 8.479055] systemd[1]: Started PostgreSQL Server.1040gitlab # [ 8.482635] systemd[1]: Starting PostgreSQL Setup Scripts...1041gitlab # [ 8.627783] postgresql-setup-start[877]: CREATE DATABASE1042gitlab # [ 8.649925] postgresql-setup-start[882]: CREATE ROLE1043gitlab # [ 8.658588] postgresql-setup-start[884]: ALTER DATABASE1044gitlab # [ 8.677456] postgresql-setup-start[889]: CREATE ROLE1045gitlab # [ 8.680589] systemd[1]: Finished PostgreSQL Setup Scripts.1046gitlab # [ 8.681376] systemd[1]: Reached target PostgreSQL.1047gitlab # [ 8.682828] systemd[1]: Starting gitlab-postgresql.service...1048gitlab # [ 8.774403] gitlab-postgresql-start[897]: CREATE DATABASE1049gitlab # [ 8.805180] gitlab-postgresql-start[901]: CREATE EXTENSION1050gitlab # [ 8.889467] gitlab-postgresql-start[907]: CREATE EXTENSION1051gitlab # [ 8.891853] systemd[1]: Finished gitlab-postgresql.service.1052gitlab # [ 8.963965] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[917]: mkdir -p /var/gitlab/state/home/.ssh: OK1053gitlab # [ 8.965157] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[917]: chmod 700 /var/gitlab/state/home/.ssh: OK1054gitlab # [ 8.998925] systemd[1]: Finished gitlab-config.service.1055gitlab # [ 9.001912] systemd[1]: Started gitaly.service.1056gitlab # [ 9.003802] systemd[1]: Starting gitlab-db-config.service...1057gitlab # [ 9.400290] dhcpcd[773]: eth0: soliciting an IPv6 router1058gitlab # [ 9.401632] dhcpcd[773]: eth0: Router Advertisement from fe80::21059gitlab # [ 9.402390] dhcpcd[773]: eth0: adding address fec0::5054:ff:fe12:3456/641060gitlab # [ 9.403221] dhcpcd[773]: eth0: adding route to fec0::/641061gitlab # [ 9.404157] dhcpcd[773]: eth0: adding default route via fe80::21062gitlab # [ 9.515131] gitaly[932]: time="2026-09-01T10:31:45.115Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=9321063gitlab # [ 9.515954] gitaly[932]: time="2026-09-01T10:31:45.115Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=9321064gitlab # [ 9.517157] gitaly[932]: time="2026-09-01T10:31:45.115Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=9321065gitlab # [ 9.517760] gitaly[932]: time="2026-09-01T10:31:45.115Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=9321066gitlab # [ 9.518726] gitaly[932]: time="2026-09-01T10:31:45.115Z" level=info msg="Starting Gitaly" pid=932 version=1067gitlab # [ 9.710749] gitaly[932]: time="2026-09-01T10:31:45.311Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=181 pid=9321068gitlab # [ 9.711439] gitaly[932]: time="2026-09-01T10:31:45.311Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=9321069gitlab # [ 9.712286] gitaly[932]: time="2026-09-01T10:31:45.311Z" level=info msg="finished initializing command factory" duration_ms=0 pid=9321070gitlab # [ 9.712837] gitaly[932]: time="2026-09-01T10:31:45.311Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-932/git-exec-3305437723.d/git pid=9321071gitlab # [ 9.718777] gitaly[932]: time="2026-09-01T10:31:45.319Z" level=info msg="finished detecting git version" duration_ms=7 pid=9321072gitlab # [ 9.719725] gitaly[932]: time="2026-09-01T10:31:45.319Z" level=info msg="using Git version" pid=932 version=2.53.g1073gitlab # [ 9.727156] gitaly[932]: time="2026-09-01T10:31:45.327Z" level=info msg="clearing disk cache object folder" pid=932 storage=default1074gitlab # [ 9.728163] gitaly[932]: time="2026-09-01T10:31:45.328Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3635464636 pid=932 storage=default1075gitlab # [ 9.729169] gitaly[932]: time="2026-09-01T10:31:45.328Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3635464636 pid=932 storage=default1076gitlab # [ 9.730238] gitaly[932]: time="2026-09-01T10:31:45.328Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=9321077gitlab # [ 9.731054] gitaly[932]: time="2026-09-01T10:31:45.328Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=9321078gitlab # [ 9.731936] gitaly[932]: time="2026-09-01T10:31:45.328Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3635464636 clear_duration_ms=0 pid=932 storage=default1079gitlab # [ 9.809376] gitaly[932]: time="2026-09-01T10:31:45.409Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=9321080gitlab # [ 9.813072] gitaly[932]: time="2026-09-01T10:31:45.410Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=932 schema=unix1081gitlab # [ 9.816630] gitaly[932]: time="2026-09-01T10:31:45.410Z" level=info msg="listening at address" address=/run/gitaly/gitaly-932/sock.d/intern pid=932 schema=unix1082gitlab # [ 9.819962] gitaly[932]: time="2026-09-01T10:31:45.410Z" level=info msg="finished tempdir cleaner walk" pid=932 storage=default time_ms=01083gitlab # [ 9.822950] gitaly[932]: time="2026-09-01T10:31:45.411Z" level=info msg="counting repositories" pid=9321084gitlab # [ 9.825399] gitaly[932]: time="2026-09-01T10:31:45.411Z" level=info msg="starting to count repositories" pid=932 storage_path=/var/gitlab/state/repositories1085gitlab # [ 9.828600] gitaly[932]: time="2026-09-01T10:31:45.411Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=932 storage_path=/var/gitlab/state/repositories1086gitlab # [ 9.832649] gitaly[932]: time="2026-09-01T10:31:45.411Z" level=info msg="completed counting all repositories" counting_duration=0 pid=9321087gitlab # [ 9.834883] gitaly[932]: time="2026-09-01T10:31:45.412Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9321088gitlab # [ 9.837714] gitaly[932]: time="2026-09-01T10:31:45.412Z" level=info msg="Started Gitaly" duration_ms=296 pid=9321089gitlab # [ 9.838189] gitaly[932]: time="2026-09-01T10:31:45.412Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9321090gitlab # [ 9.839150] gitaly[932]: time="2026-09-01T10:31:45.413Z" level=info msg="maintenance: daily scheduled" pid=932 scheduled="2026-09-01 12:00:00 +0000 UTC"1091gitlab # [ 10.155827] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[935]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1092gitlab # [ 10.156496] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[935]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1093gitlab # [ 10.170648] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[935]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1094gitlab # [ 11.649722] dhcpcd[773]: eth0: leased 10.0.2.15 for 86400 seconds1095gitlab # [ 11.650486] dhcpcd[773]: eth0: adding route to 10.0.2.0/241096gitlab # [ 11.651261] dhcpcd[773]: eth0: adding default route via 10.0.2.21097gitlab # [ 11.715469] systemd[1]: Started DHCP Client.1098gitlab # [ 11.717210] systemd[1]: Reached target Network is Online.1099gitlab # [ 13.211786] gitaly[932]: time="2026-09-01T10:31:48.811Z" level=info msg="License database preloaded" duration_ms=3696 pid=9321100gitlab # [ 49.113640] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[935]: W, [2026-09-01T10:32:24.713934 #935] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1101gitlab # [ 51.564217] postgres[1406]: [1406] ERROR: relation "feature_gates" does not exist at character 5231102gitlab # [ 51.564684] postgres[1406]: [1406] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1103gitlab # [ 51.565271] postgres[1406]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1104gitlab # [ 51.565836] postgres[1406]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1105gitlab # [ 51.567122] postgres[1406]: attidentity AS identity,1106gitlab # [ 51.568136] postgres[1406]: attgenerated as attgenerated1107gitlab # [ 51.569117] postgres[1406]: FROM pg_attribute a1108gitlab # [ 51.570078] postgres[1406]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1109gitlab # [ 51.570935] postgres[1406]: LEFT JOIN pg_type t ON a.atttypid = t.oid1110gitlab # [ 51.572181] postgres[1406]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1111gitlab # [ 51.573058] postgres[1406]: WHERE a.attrelid = '"feature_gates"'::regclass1112gitlab # [ 51.573879] postgres[1406]: AND a.attnum > 0 AND NOT a.attisdropped1113gitlab # [ 51.574957] postgres[1406]: ORDER BY a.attnum1114gitlab # [ 51.576080] postgres[1406]:1115gitlab # [ 55.701569] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1448]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1116gitlab # [ 55.702235] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1448]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1117gitlab # [ 57.772373] gitlab-workhorse[832]: time="2026-09-01T10:32:33Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1118gitlab # [ 67.781438] gitlab-workhorse[832]: time="2026-09-01T10:32:43Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1119gitlab # [ 74.875805] postgres[1550]: [1550] ERROR: relation "table_without_models" does not exist1120gitlab # [ 74.876117] postgres[1550]: [1550] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1121gitlab # [ 77.791150] gitlab-workhorse[832]: time="2026-09-01T10:32:53Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1122gitlab # [ 87.800192] gitlab-workhorse[832]: time="2026-09-01T10:33:03Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1123gitlab # [ 91.300529] systemd[1]: Finished gitlab-db-config.service.1124gitlab # [ 91.304191] systemd[1]: Started gitlab-sidekiq.service.1125gitlab # [ 91.306133] systemd[1]: Starting gitlab.service...1126gitlab # [ 91.856842] bundle[1772]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1127gitlab # [ 91.857284] bundle[1772]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1128gitlab # [ 91.859798] bundle[1772]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1129gitlab # [ 96.208402] sidekiq-cluster[1771]: {"severity":"INFO","time":"2026-09-01T10:33:11.783Z","message":"Starting cluster with 1 processes"}1130gitlab # [ 96.552490] bundle[1772]: {"timestamp":"2026-09-01T10:33:12.123Z","pid":1772,"message":"* Enabling systemd notification integration"}1131gitlab # [ 96.553351] bundle[1772]: {"timestamp":"2026-09-01T10:33:12.152Z","pid":1772,"message":"Puma starting in cluster mode..."}1132gitlab # [ 96.554068] bundle[1772]: {"timestamp":"2026-09-01T10:33:12.152Z","pid":1772,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1133gitlab # [ 96.554723] bundle[1772]: {"timestamp":"2026-09-01T10:33:12.153Z","pid":1772,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1134gitlab # [ 96.555635] bundle[1772]: {"timestamp":"2026-09-01T10:33:12.153Z","pid":1772,"message":"* Min threads: 0"}1135gitlab # [ 96.556574] bundle[1772]: {"timestamp":"2026-09-01T10:33:12.153Z","pid":1772,"message":"* Max threads: 4"}1136gitlab # [ 96.557363] bundle[1772]: {"timestamp":"2026-09-01T10:33:12.153Z","pid":1772,"message":"* Environment: production"}1137gitlab # [ 96.558072] bundle[1772]: {"timestamp":"2026-09-01T10:33:12.153Z","pid":1772,"message":"* Master PID: 1772"}1138gitlab # [ 96.558785] bundle[1772]: {"timestamp":"2026-09-01T10:33:12.153Z","pid":1772,"message":"* Workers: 2"}1139gitlab # [ 96.559705] bundle[1772]: {"timestamp":"2026-09-01T10:33:12.153Z","pid":1772,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1140gitlab # [ 96.560654] bundle[1772]: {"timestamp":"2026-09-01T10:33:12.153Z","pid":1772,"message":"* Preloading application"}1141gitlab # [ 97.801181] gitlab-workhorse[832]: time="2026-09-01T10:33:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1142gitlab # [ 107.810954] gitlab-workhorse[832]: time="2026-09-01T10:33:23Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1143gitlab # [ 117.819995] gitlab-workhorse[832]: time="2026-09-01T10:33:33Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1144gitlab # [ 127.821334] gitlab-workhorse[832]: time="2026-09-01T10:33:43Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1145gitlab # [ 128.048353] bundle[1772]: W, [2026-09-01T10:33:43.648379 #1772] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1146gitlab # [ 132.574650] sidekiq-cluster[1811]: 2026-09-01T10:33:48.174Z pid=1811 tid=wqj INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1811, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>433852416}1147gitlab # [ 132.837211] sidekiq-cluster[1811]: W, [2026-09-01T10:33:48.437562 #1811] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1148gitlab # [ 132.897303] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:48.497Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1149gitlab # [ 135.856432] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.456Z","message":"GitLab reliable fetch activated!"}1150gitlab # [ 135.857223] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.456Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1151gitlab # [ 135.885134] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.485Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1152gitlab # [ 135.887588] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.488Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1153gitlab # [ 135.890536] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.491Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1154gitlab # [ 135.894776] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.495Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1155gitlab # [ 135.898432] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.499Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1156gitlab # [ 135.901811] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.502Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1157gitlab # [ 135.905363] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.505Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1158gitlab # [ 135.908199] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.508Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1159gitlab # [ 135.910785] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.511Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1160gitlab # [ 135.913556] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.514Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1161gitlab # [ 135.916644] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.517Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1162gitlab # [ 135.921898] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.522Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1163gitlab # [ 135.925799] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.526Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1164gitlab # [ 135.928798] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.529Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1165gitlab # [ 135.932094] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.532Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1166gitlab # [ 135.935405] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.535Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1167gitlab # [ 135.939109] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.539Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1168gitlab # [ 135.942830] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.543Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1169gitlab # [ 135.946232] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.546Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1170gitlab # [ 135.949495] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.550Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1171gitlab # [ 135.953071] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.553Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1172gitlab # [ 135.961050] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.561Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1173gitlab # [ 135.968370] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.568Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1174gitlab # [ 135.974516] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.575Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1175gitlab # [ 135.977463] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.578Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1176gitlab # [ 135.980910] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.581Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1177gitlab # [ 135.983929] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.584Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1178gitlab # [ 135.987277] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.587Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1179gitlab # [ 135.991060] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.591Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1180gitlab # [ 135.994508] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.595Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1181gitlab # [ 135.997319] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.597Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1182gitlab # [ 136.000773] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.601Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1183gitlab # [ 136.004467] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.605Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1184gitlab # [ 136.008293] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.608Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1185gitlab # [ 136.011657] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.612Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1186gitlab # [ 136.014973] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.615Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1187gitlab # [ 136.018105] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.618Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1188gitlab # [ 136.021912] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.622Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1189gitlab # [ 136.025940] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.626Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1190gitlab # [ 136.030245] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.630Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1191gitlab # [ 136.033711] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.634Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1192gitlab # [ 136.037944] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.638Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1193gitlab # [ 136.042573] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.643Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1194gitlab # [ 136.046828] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.647Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1195gitlab # [ 136.051790] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.652Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1196gitlab # [ 136.056449] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.657Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1197gitlab # [ 136.060561] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.661Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1198gitlab # [ 136.064948] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.665Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1199gitlab # [ 136.069993] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.670Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1200gitlab # [ 136.073223] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.673Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1201gitlab # [ 136.076976] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.677Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1202gitlab # [ 136.080496] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.681Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1203gitlab # [ 136.089086] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.689Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1204gitlab # [ 136.092496] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.693Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1205gitlab # [ 136.097617] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.698Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1206gitlab # [ 136.102434] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.702Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1207gitlab # [ 136.107121] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.707Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1208gitlab # [ 136.113704] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.714Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1209gitlab # [ 136.117919] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.718Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1210gitlab # [ 136.122292] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.722Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1211gitlab # [ 136.125889] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.726Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1212gitlab # [ 136.129249] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.729Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1213gitlab # [ 136.134051] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.734Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1214gitlab # [ 136.138979] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.739Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1215gitlab # [ 136.143282] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.743Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1216gitlab # [ 136.147787] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.748Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1217gitlab # [ 136.150814] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.751Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1218gitlab # [ 136.157197] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.757Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1219gitlab # [ 136.160917] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.761Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1220gitlab # [ 136.164946] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.765Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1221gitlab # [ 136.168810] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.769Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1222gitlab # [ 136.172643] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.773Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1223gitlab # [ 136.177198] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.777Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1224gitlab # [ 136.233190] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.833Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1225gitlab # [ 136.236249] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.836Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1226gitlab # [ 136.240872] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.841Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1227gitlab # [ 136.244798] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.845Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1228gitlab # [ 136.250072] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.850Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1229gitlab # [ 136.254061] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.854Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1230gitlab # [ 136.258739] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.859Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1231gitlab # [ 136.262825] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.863Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1232gitlab # [ 136.266923] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.867Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1233gitlab # [ 136.271841] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.872Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1234gitlab # [ 136.276036] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.876Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1235gitlab # [ 136.280651] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.881Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1236gitlab # [ 136.285344] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.885Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1237gitlab # [ 136.289470] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.890Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1238gitlab # [ 136.294072] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.894Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1239gitlab # [ 136.297873] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.898Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1240gitlab # [ 136.302293] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.902Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1241gitlab # [ 136.305581] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.906Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1242gitlab # [ 136.309993] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.910Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1243gitlab # [ 136.312641] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.913Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1244gitlab # [ 136.316918] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.917Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1245gitlab # [ 136.321784] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.922Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1246gitlab # [ 136.326941] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.927Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1247gitlab # [ 136.330231] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.930Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1248gitlab # [ 136.333785] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.934Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1249gitlab # [ 136.344323] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.944Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1250gitlab # [ 136.347559] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.948Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1251gitlab # [ 136.351056] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.951Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1252gitlab # [ 136.355450] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.956Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1253gitlab # [ 136.359982] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.960Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1254gitlab # [ 136.365608] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.965Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1255gitlab # [ 136.369372] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.969Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1256gitlab # [ 136.372323] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.972Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1257gitlab # [ 136.375723] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.976Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1258gitlab # [ 136.380219] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.980Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1259gitlab # [ 136.383343] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:33:51.983Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1260gitlab # [ 137.823225] gitlab-workhorse[832]: time="2026-09-01T10:33:53Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1261gitlab # [ 147.832210] gitlab-workhorse[832]: time="2026-09-01T10:34:03Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1262gitlab # [ 157.841941] gitlab-workhorse[832]: time="2026-09-01T10:34:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1263gitlab # [ 164.248458] postgres[2439]: [2439] ERROR: relation "table_without_models" does not exist1264gitlab # [ 164.249104] postgres[2439]: [2439] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1265gitlab # [ 165.529175] postgres[2459]: [2459] ERROR: relation "table_without_models" does not exist1266gitlab # [ 165.529869] postgres[2459]: [2459] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1267gitlab # [ 167.851118] gitlab-workhorse[832]: time="2026-09-01T10:34:23Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1268gitlab # [ 167.859557] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:23.456Z","message":"Booted Rails 7.2.3.1 application in production environment"}1269gitlab # [ 167.860070] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:23.460Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1270gitlab # [ 167.860855] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:23.460Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1271gitlab # [ 167.861938] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:23.460Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1272gitlab # [ 167.870495] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:23.471Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1273gitlab # [ 169.114962] bundle[1772]: {"timestamp":"2026-09-01T10:34:24.715Z","pid":1772,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1274gitlab # [ 169.115961] bundle[1772]: {"timestamp":"2026-09-01T10:34:24.715Z","pid":1772,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1275gitlab # [ 169.116833] bundle[1772]: {"timestamp":"2026-09-01T10:34:24.715Z","pid":1772,"message":"! #\u003cThread:0x00007229c01dcde0 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1276gitlab # [ 169.117559] bundle[1772]: {"timestamp":"2026-09-01T10:34:24.715Z","pid":1772,"message":"! #\u003cThread:0x00007229c038b650 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1277gitlab # [ 169.118323] bundle[1772]: {"timestamp":"2026-09-01T10:34:24.715Z","pid":1772,"message":"Use Ctrl-C to stop"}1278gitlab # [ 169.134101] bundle[1772]: {"timestamp":"2026-09-01T10:34:24.734Z","pid":1772,"message":"Sending status to systemd every 1.0 sec"}1279gitlab # [ 169.583740] bundle[1772]: {"timestamp":"2026-09-01T10:34:25.184Z","pid":1772,"message":"- Worker 0 (PID: 2516) booted in 0.43s, phase: 0"}1280gitlab # [ 169.591430] bundle[1772]: {"timestamp":"2026-09-01T10:34:25.191Z","pid":1772,"message":"- Worker 1 (PID: 2520) booted in 0.42s, phase: 0"}1281gitlab # [ 169.602067] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:25.202Z","message":"Cleaning working queues"}1282gitlab # [ 169.605861] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:25.205Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"8fd1955103446832bf14c224","created_at":"2026-09-01T10:33:06.315Z","trace_propagation_headers":{"sentry-trace":"998954d13a5546eebd033889d19df55f-4e83cee015094982","baggage":"sentry-trace_id=998954d13a5546eebd033889d19df55f,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"54d78ae7ea4cbb8e598577f4b1519409","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-01T10:33:06.331Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-8fd1955103446832bf14c224: start","job_status":"start","queue_duration_s":78.87414,"scheduling_latency_s":78.874212}1283gitlab # [ 169.757432] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:25.357Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"8fd1955103446832bf14c224","created_at":"2026-09-01T10:33:06.315Z","trace_propagation_headers":{"sentry-trace":"998954d13a5546eebd033889d19df55f-4e83cee015094982","baggage":"sentry-trace_id=998954d13a5546eebd033889d19df55f,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"54d78ae7ea4cbb8e598577f4b1519409","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-01T10:33:06.331Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-8fd1955103446832bf14c224: done: 0.152585 sec","job_status":"done","queue_duration_s":78.87414,"scheduling_latency_s":78.874212,"redis_calls":26,"redis_duration_s":0.006054,"redis_read_bytes":914,"redis_write_bytes":2887,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.00313,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001845,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001079,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.077915,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.084115999,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.152585,"completed_at":"2026-09-01T10:34:25.357Z","load_balancing_strategy":"primary","db_duration_s":0.039424,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1284gitlab # [ 169.891054] systemd[1]: Started gitlab.service.1285gitlab # [ 169.892610] systemd[1]: Reached target Common target for all GitLab services..1286gitlab # [ 169.895491] systemd[1]: Starting setup-gitlab.service...1287gitlab # [ 169.913278] setup-gitlab-start[2560]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1288gitlab # [ 172.486816] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:34:28 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 25521289gitlab # [ 172.490471] setup-gitlab-start[2560]: + TOKEN=nixbot-test-token-of-20-chars1290gitlab # [ 172.491362] setup-gitlab-start[2560]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1291gitlab # [ 172.513924] runuser[2581]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1292gitlab # [ 173.035469] setup-gitlab-start[2583]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1293gitlab # [ 173.035850] setup-gitlab-start[2583]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1294gitlab # [ 173.038664] setup-gitlab-start[2583]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1295gitlab # [ 173.887452] setup-gitlab-start[2583]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1296gitlab # [ 173.888250] setup-gitlab-start[2583]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1297gitlab # [ 173.891506] setup-gitlab-start[2583]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1298gitlab # [ 180.912473] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:36.512Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"3d45bf697e8976a3d909ec0c","created_at":"2026-09-01T10:34:36.488Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"071acfe1df4bd311092452dc31965532","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:36.510Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-3d45bf697e8976a3d909ec0c: start","job_status":"start","queue_duration_s":0.002192,"scheduling_latency_s":0.002232}1299gitlab # [ 181.004795] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:36.604Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"3d45bf697e8976a3d909ec0c","created_at":"2026-09-01T10:34:36.488Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"071acfe1df4bd311092452dc31965532","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:36.510Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-3d45bf697e8976a3d909ec0c: done: 0.092642 sec","job_status":"done","queue_duration_s":0.002192,"scheduling_latency_s":0.002232,"redis_calls":12,"redis_duration_s":0.030468,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.020938,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006493,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003037,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020246,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000801219,"duration_s":0.092642,"completed_at":"2026-09-01T10:34:36.604Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00244,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1300gitlab # [ 181.065336] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:36.609Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"57a261149e9d43c260411499","created_at":"2026-09-01T10:34:36.543Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d70bf4b9a3b2f36adef977ee0421bfef","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:36.558Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-57a261149e9d43c260411499: start","job_status":"start","queue_duration_s":0.050687,"scheduling_latency_s":0.050729}1301gitlab # [ 181.108950] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:36.709Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"57a261149e9d43c260411499","created_at":"2026-09-01T10:34:36.543Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d70bf4b9a3b2f36adef977ee0421bfef","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:36.558Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-57a261149e9d43c260411499: done: 0.099757 sec","job_status":"done","queue_duration_s":0.050687,"scheduling_latency_s":0.050729,"redis_calls":12,"redis_duration_s":0.027379,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002917,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.018611,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005851,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01937,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.002237436,"exclusive_lock_hold_duration_s":0.042849021,"duration_s":0.099757,"completed_at":"2026-09-01T10:34:36.708Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.039297,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1302gitlab # [ 181.143621] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:36.714Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"c9dc82cdb722191d76da710d","created_at":"2026-09-01T10:34:36.586Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be0c4a5f9037dce2ac6e0cc6ea66ad88","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:36.621Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-c9dc82cdb722191d76da710d: start","job_status":"start","queue_duration_s":0.093408,"scheduling_latency_s":0.093444}1303gitlab # [ 181.155836] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:36.756Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1304gitlab # [ 181.177229] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:36.777Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"c9dc82cdb722191d76da710d","created_at":"2026-09-01T10:34:36.586Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be0c4a5f9037dce2ac6e0cc6ea66ad88","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:36.621Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-c9dc82cdb722191d76da710d: done: 0.06243 sec","job_status":"done","queue_duration_s":0.093408,"scheduling_latency_s":0.093444,"redis_calls":10,"redis_duration_s":0.020852,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005721,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.011068,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004063,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018875,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000644216,"duration_s":0.06243,"completed_at":"2026-09-01T10:34:36.777Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001628,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1305gitlab # [ 181.251292] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:36.779Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"9a0bf86839ee184982de4b8e","created_at":"2026-09-01T10:34:36.638Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dcbdae834d5a4c57fce0696749977253","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-01T10:34:36.654Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-9a0bf86839ee184982de4b8e: start","job_status":"start","queue_duration_s":0.125474,"scheduling_latency_s":0.125512}1306gitlab # [ 181.252712] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:36.842Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"9a0bf86839ee184982de4b8e","created_at":"2026-09-01T10:34:36.638Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dcbdae834d5a4c57fce0696749977253","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-01T10:34:36.654Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-9a0bf86839ee184982de4b8e: done: 0.062612 sec","job_status":"done","queue_duration_s":0.125474,"scheduling_latency_s":0.125512,"redis_calls":10,"redis_duration_s":0.012006,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002631,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007509,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001866,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026526,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001510527,"duration_s":0.062612,"completed_at":"2026-09-01T10:34:36.842Z","load_balancing_strategy":"primary","db_duration_s":0.010233,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1307gitlab # [ 181.316884] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:36.848Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"3a049689b43c860a4cb2bc7e","created_at":"2026-09-01T10:34:36.664Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2666edef581313b0f44d3ddfe73a364","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:36.673Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-3a049689b43c860a4cb2bc7e: start","job_status":"start","queue_duration_s":0.17478,"scheduling_latency_s":0.174819}1308gitlab # [ 181.342137] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:36.942Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"3a049689b43c860a4cb2bc7e","created_at":"2026-09-01T10:34:36.664Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2666edef581313b0f44d3ddfe73a364","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:36.673Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-3a049689b43c860a4cb2bc7e: done: 0.094076 sec","job_status":"done","queue_duration_s":0.17478,"scheduling_latency_s":0.174819,"redis_calls":12,"redis_duration_s":0.030069,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003091,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.021477,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005501,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021487,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001306591,"exclusive_lock_hold_duration_s":0.00521463,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.094076,"completed_at":"2026-09-01T10:34:36.942Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004763,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1309gitlab # [ 181.353368] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:36.944Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"deda3abd2a7f0e2519afdac1","created_at":"2026-09-01T10:34:36.693Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"251fdcc48ca846c0b2a92bac315847d4","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:36.711Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-deda3abd2a7f0e2519afdac1: start","job_status":"start","queue_duration_s":0.233409,"scheduling_latency_s":0.233446}1310gitlab # [ 182.478796] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:38.073Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"deda3abd2a7f0e2519afdac1","created_at":"2026-09-01T10:34:36.693Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"251fdcc48ca846c0b2a92bac315847d4","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:36.711Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-deda3abd2a7f0e2519afdac1: done: 1.12864 sec","job_status":"done","queue_duration_s":0.233409,"scheduling_latency_s":0.233446,"redis_calls":565,"redis_duration_s":0.593006,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001447,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003983,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.587576,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.525566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000642261,"duration_s":1.12864,"completed_at":"2026-09-01T10:34:38.073Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001186,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1311gitlab # [ 182.552351] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:38.150Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c9a5f2d6ce2f29076bcc9b54","created_at":"2026-09-01T10:34:36.718Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c1c56db9c07d1f7712f4cf7bf681fe25","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:36.736Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c9a5f2d6ce2f29076bcc9b54: start","job_status":"start","queue_duration_s":1.413593,"scheduling_latency_s":1.413648}1312gitlab # [ 182.609453] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:38.209Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c9a5f2d6ce2f29076bcc9b54","created_at":"2026-09-01T10:34:36.718Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c1c56db9c07d1f7712f4cf7bf681fe25","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:36.736Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c9a5f2d6ce2f29076bcc9b54: done: 0.059053 sec","job_status":"done","queue_duration_s":1.413593,"scheduling_latency_s":1.413648,"redis_calls":10,"redis_duration_s":0.005484999999999999,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001406,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002655,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001424,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021611,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000560127,"duration_s":0.059053,"completed_at":"2026-09-01T10:34:38.209Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003709,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1313gitlab # [ 182.672857] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:38.273Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"789a670a531aa9c10220ec9d","created_at":"2026-09-01T10:34:36.757Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"537653540c4e4b0d0ce55001f3cf6e5e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:36.768Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-789a670a531aa9c10220ec9d: start","job_status":"start","queue_duration_s":1.504854,"scheduling_latency_s":1.504898}1314gitlab # [ 183.237933] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:38.837Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"789a670a531aa9c10220ec9d","created_at":"2026-09-01T10:34:36.757Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"537653540c4e4b0d0ce55001f3cf6e5e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:36.768Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-789a670a531aa9c10220ec9d: done: 0.56423 sec","job_status":"done","queue_duration_s":1.504854,"scheduling_latency_s":1.504898,"redis_calls":14,"redis_duration_s":0.009316999999999999,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001885,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005038,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002394,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.127,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.127,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.308471,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001032813,"exclusive_lock_hold_duration_s":0.509682655,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.56423,"completed_at":"2026-09-01T10:34:38.837Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.175798,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1315gitlab # [ 183.388432] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:38.844Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"f7f1f3d57ecfd960d2df79e8","created_at":"2026-09-01T10:34:36.785Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d99aa825739b44b00b14e4bacbc33f79","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:36.805Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-f7f1f3d57ecfd960d2df79e8: start","job_status":"start","queue_duration_s":2.039201,"scheduling_latency_s":2.039257}1316gitlab # [ 183.414397] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:38.953Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"f7f1f3d57ecfd960d2df79e8","created_at":"2026-09-01T10:34:36.785Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d99aa825739b44b00b14e4bacbc33f79","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:36.805Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-f7f1f3d57ecfd960d2df79e8: done: 0.108976 sec","job_status":"done","queue_duration_s":2.039201,"scheduling_latency_s":2.039257,"redis_calls":13,"redis_duration_s":0.020485,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004467,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.011006,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005012,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041396,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.021294326000000002,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.108976,"completed_at":"2026-09-01T10:34:38.953Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007162,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1317gitlab # [ 183.491429] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:38.956Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"8b352be1141da63a2edc2f07","created_at":"2026-09-01T10:34:36.829Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cabc04973e69e2d67e2c96ce864e5565","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-01T10:34:36.847Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-8b352be1141da63a2edc2f07: start","job_status":"start","queue_duration_s":2.108863,"scheduling_latency_s":2.108906}1318gitlab # [ 183.511599] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:38.979Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1319gitlab # [ 183.518138] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:39.019Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"8b352be1141da63a2edc2f07","created_at":"2026-09-01T10:34:36.829Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cabc04973e69e2d67e2c96ce864e5565","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-01T10:34:36.847Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-8b352be1141da63a2edc2f07: done: 0.063452 sec","job_status":"done","queue_duration_s":2.108863,"scheduling_latency_s":2.108906,"redis_calls":10,"redis_duration_s":0.004517,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000936,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002373,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001208,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017724,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000527162,"duration_s":0.063452,"completed_at":"2026-09-01T10:34:39.019Z","load_balancing_strategy":"primary","db_duration_s":0.000808,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1320gitlab # [ 183.639370] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:39.023Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"dffa03e36328ff05d756825c","created_at":"2026-09-01T10:34:36.878Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a306fbb6d452bbc8a8d2446440fed96f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-01T10:34:36.900Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-dffa03e36328ff05d756825c: start","job_status":"start","queue_duration_s":2.123016,"scheduling_latency_s":2.123059}1321gitlab # [ 183.669939] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:39.078Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1322gitlab # [ 183.679487] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:39.092Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"dffa03e36328ff05d756825c","created_at":"2026-09-01T10:34:36.878Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a306fbb6d452bbc8a8d2446440fed96f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-01T10:34:36.900Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-dffa03e36328ff05d756825c: done: 0.068621 sec","job_status":"done","queue_duration_s":2.123016,"scheduling_latency_s":2.123059,"redis_calls":10,"redis_duration_s":0.009969,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001371,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00538,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003218,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02175,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000782502,"duration_s":0.068621,"completed_at":"2026-09-01T10:34:39.092Z","load_balancing_strategy":"primary","db_duration_s":0.001061,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1323gitlab # [ 183.746179] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:39.095Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"673133d39ae976337f16d955","created_at":"2026-09-01T10:34:36.937Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"24eb71e40c980714cdd557c12117dc25","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:36.955Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-673133d39ae976337f16d955: start","job_status":"start","queue_duration_s":2.139419,"scheduling_latency_s":2.139463}1324gitlab # [ 183.747124] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:39.197Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"673133d39ae976337f16d955","created_at":"2026-09-01T10:34:36.937Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"24eb71e40c980714cdd557c12117dc25","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:36.955Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-673133d39ae976337f16d955: done: 0.102346 sec","job_status":"done","queue_duration_s":2.139419,"scheduling_latency_s":2.139463,"redis_calls":10,"redis_duration_s":0.008047,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001258,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005221,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001568,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039581,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001171098,"duration_s":0.102346,"completed_at":"2026-09-01T10:34:39.197Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016893,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1325gitlab # [ 183.767235] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:39.202Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"64c95f614130733cfd30b55c","created_at":"2026-09-01T10:34:36.964Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"14a6ad5a623d8910ad2070c13a75ebea","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.005Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-64c95f614130733cfd30b55c: start","job_status":"start","queue_duration_s":2.196039,"scheduling_latency_s":2.196345}1326gitlab # [ 183.788773] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:39.262Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"64c95f614130733cfd30b55c","created_at":"2026-09-01T10:34:36.964Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"14a6ad5a623d8910ad2070c13a75ebea","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.005Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-64c95f614130733cfd30b55c: done: 0.060838 sec","job_status":"done","queue_duration_s":2.196039,"scheduling_latency_s":2.196345,"redis_calls":10,"redis_duration_s":0.008258999999999999,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001752,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004541,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001966,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000613486,"duration_s":0.060838,"completed_at":"2026-09-01T10:34:39.262Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001122,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1327gitlab # [ 183.912312] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:39.266Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"52624dc9f36aa471b1ebb8a3","created_at":"2026-09-01T10:34:37.038Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"479af40aff058c1d65b2f10030be71ff","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.048Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-52624dc9f36aa471b1ebb8a3: start","job_status":"start","queue_duration_s":2.217256,"scheduling_latency_s":2.217348}1328gitlab # [ 183.913848] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:39.300Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1329gitlab # [ 183.916195] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:39.336Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"52624dc9f36aa471b1ebb8a3","created_at":"2026-09-01T10:34:37.038Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"479af40aff058c1d65b2f10030be71ff","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.048Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-52624dc9f36aa471b1ebb8a3: done: 0.070714 sec","job_status":"done","queue_duration_s":2.217256,"scheduling_latency_s":2.217348,"redis_calls":10,"redis_duration_s":0.007428,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00104,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004506,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001882,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028449,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001795759,"duration_s":0.070714,"completed_at":"2026-09-01T10:34:39.336Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00177,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1330gitlab # [ 184.099827] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:39.699Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"21ad2ede47289754294b4890","created_at":"2026-09-01T10:34:37.056Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3fb7ae9c9f191f7dd7db917b2a551c12","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.066Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-21ad2ede47289754294b4890: start","job_status":"start","queue_duration_s":2.632998,"scheduling_latency_s":2.633067}1331gitlab # [ 184.175551] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:39.776Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1332gitlab # [ 184.189835] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:39.789Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"21ad2ede47289754294b4890","created_at":"2026-09-01T10:34:37.056Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3fb7ae9c9f191f7dd7db917b2a551c12","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.066Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-21ad2ede47289754294b4890: done: 0.090213 sec","job_status":"done","queue_duration_s":2.632998,"scheduling_latency_s":2.633067,"redis_calls":10,"redis_duration_s":0.009035999999999999,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000837,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004042,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004157,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017732,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000762108,"duration_s":0.090213,"completed_at":"2026-09-01T10:34:39.789Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000845,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1333gitlab # [ 184.233055] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:39.793Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"b0ef47421f7759428c3d10e1","created_at":"2026-09-01T10:34:37.083Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"688e04a2d83412f6f6fe8e16e521f1c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.096Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-b0ef47421f7759428c3d10e1: start","job_status":"start","queue_duration_s":2.696308,"scheduling_latency_s":2.696349}1334gitlab # [ 184.234390] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:39.817Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1335gitlab # [ 184.235250] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:39.830Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"b0ef47421f7759428c3d10e1","created_at":"2026-09-01T10:34:37.083Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"688e04a2d83412f6f6fe8e16e521f1c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.096Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-b0ef47421f7759428c3d10e1: done: 0.037132 sec","job_status":"done","queue_duration_s":2.696308,"scheduling_latency_s":2.696349,"redis_calls":10,"redis_duration_s":0.0040290000000000005,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000626,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002092,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001311,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017511,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000434692,"duration_s":0.037132,"completed_at":"2026-09-01T10:34:39.830Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001066,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1336gitlab # [ 184.353134] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:39.953Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"8b13a7b7f50c7e45694eac0b","created_at":"2026-09-01T10:34:37.112Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bb9ee3db9130488729f27fc3faf21aad","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.120Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-8b13a7b7f50c7e45694eac0b: start","job_status":"start","queue_duration_s":2.832121,"scheduling_latency_s":2.832168}1337gitlab # [ 184.415488] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.015Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"8b13a7b7f50c7e45694eac0b","created_at":"2026-09-01T10:34:37.112Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bb9ee3db9130488729f27fc3faf21aad","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.120Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-8b13a7b7f50c7e45694eac0b: done: 0.062475 sec","job_status":"done","queue_duration_s":2.832121,"scheduling_latency_s":2.832168,"redis_calls":10,"redis_duration_s":0.004326,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000736,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002613,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000977,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000389994,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.062475,"completed_at":"2026-09-01T10:34:40.015Z","load_balancing_strategy":"primary","db_duration_s":0.01138,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1338gitlab # [ 184.478098] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.078Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"4598be67eef91099ae2d2ef1","created_at":"2026-09-01T10:34:37.140Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a7873dbeb295b31a8594d1d1bbcd511e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.149Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-4598be67eef91099ae2d2ef1: start","job_status":"start","queue_duration_s":2.928881,"scheduling_latency_s":2.928925}1339gitlab # [ 184.541504] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.141Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"4598be67eef91099ae2d2ef1","created_at":"2026-09-01T10:34:37.140Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a7873dbeb295b31a8594d1d1bbcd511e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.149Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-4598be67eef91099ae2d2ef1: done: 0.0628 sec","job_status":"done","queue_duration_s":2.928881,"scheduling_latency_s":2.928925,"redis_calls":14,"redis_duration_s":0.004647,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00151,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001757,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00138,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000639188,"exclusive_lock_hold_duration_s":0.025157666,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.0628,"completed_at":"2026-09-01T10:34:40.140Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013788,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1340gitlab # [ 184.604532] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.204Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"912930135d16df326c8d5931","created_at":"2026-09-01T10:34:37.161Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"118eae5e07a1eac8f7996b39cb453d1b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.172Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-912930135d16df326c8d5931: start","job_status":"start","queue_duration_s":3.032298,"scheduling_latency_s":3.032341}1341gitlab # [ 184.670250] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.270Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"912930135d16df326c8d5931","created_at":"2026-09-01T10:34:37.161Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"118eae5e07a1eac8f7996b39cb453d1b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.172Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-912930135d16df326c8d5931: done: 0.065804 sec","job_status":"done","queue_duration_s":3.032298,"scheduling_latency_s":3.032341,"redis_calls":19,"redis_duration_s":0.005915999999999999,"redis_read_bytes":21,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001298,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000415,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.00295,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001253,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033032,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000593092,"duration_s":0.065804,"completed_at":"2026-09-01T10:34:40.270Z","load_balancing_strategy":"primary","db_duration_s":0.002721,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1342gitlab # [ 184.730758] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.330Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"9ce591a087da0e2ca2011d5d","created_at":"2026-09-01T10:34:37.181Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.194Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-9ce591a087da0e2ca2011d5d: start","job_status":"start","queue_duration_s":3.135813,"scheduling_latency_s":3.135861}1343gitlab # [ 184.871520] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.471Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"9ce591a087da0e2ca2011d5d","created_at":"2026-09-01T10:34:37.181Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.194Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-9ce591a087da0e2ca2011d5d: done: 0.140682 sec","job_status":"done","queue_duration_s":3.135813,"scheduling_latency_s":3.135861,"redis_calls":71,"redis_duration_s":0.017972000000000002,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002118,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000667,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.014222,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.099205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000350883,"duration_s":0.140682,"completed_at":"2026-09-01T10:34:40.471Z","load_balancing_strategy":"primary","db_duration_s":0.002345,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1344gitlab # [ 184.931201] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.531Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"974cdf79dabaf211bdb1c3e8","created_at":"2026-09-01T10:34:37.224Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2beb43f32519e17f6f69fd81c7802780","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.237Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-974cdf79dabaf211bdb1c3e8: start","job_status":"start","queue_duration_s":3.29355,"scheduling_latency_s":3.293592}1345gitlab # [ 184.996627] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.596Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"974cdf79dabaf211bdb1c3e8","created_at":"2026-09-01T10:34:37.224Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2beb43f32519e17f6f69fd81c7802780","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.237Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-974cdf79dabaf211bdb1c3e8: done: 0.065555 sec","job_status":"done","queue_duration_s":3.29355,"scheduling_latency_s":3.293592,"redis_calls":16,"redis_duration_s":0.005422,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.00209,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002288,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001044,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02752,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000581917,"duration_s":0.065555,"completed_at":"2026-09-01T10:34:40.596Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009428,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1346gitlab # [ 185.059278] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.659Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"802d08bc9e35f7b1f0146643","created_at":"2026-09-01T10:34:37.249Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ec903db4b1cfc88d1bde3d5315dd602","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.263Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-802d08bc9e35f7b1f0146643: start","job_status":"start","queue_duration_s":3.39578,"scheduling_latency_s":3.395822}1347gitlab # [ 185.115950] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.715Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"802d08bc9e35f7b1f0146643","created_at":"2026-09-01T10:34:37.249Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ec903db4b1cfc88d1bde3d5315dd602","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.263Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-802d08bc9e35f7b1f0146643: done: 0.056883 sec","job_status":"done","queue_duration_s":3.39578,"scheduling_latency_s":3.395822,"redis_calls":12,"redis_duration_s":0.00562,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000849,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001737,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003034,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021228,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0006336,"exclusive_lock_hold_duration_s":0.008837435000000001,"duration_s":0.056883,"completed_at":"2026-09-01T10:34:40.715Z","load_balancing_strategy":"primary","db_duration_s":0.004248,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1348gitlab # [ 185.148301] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.718Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"4eb214a19abbcd1042a24322","created_at":"2026-09-01T10:34:37.276Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a414ffb67233a8977442e3cab30c70a9","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.294Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-4eb214a19abbcd1042a24322: start","job_status":"start","queue_duration_s":3.423362,"scheduling_latency_s":3.423394}1349gitlab # [ 185.150568] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.750Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"4eb214a19abbcd1042a24322","created_at":"2026-09-01T10:34:37.276Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a414ffb67233a8977442e3cab30c70a9","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.294Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-4eb214a19abbcd1042a24322: done: 0.032557 sec","job_status":"done","queue_duration_s":3.423362,"scheduling_latency_s":3.423394,"redis_calls":12,"redis_duration_s":0.004887,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000738,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001326,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002191,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000632,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018384,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000507606,"duration_s":0.032557,"completed_at":"2026-09-01T10:34:40.750Z","load_balancing_strategy":"primary","db_duration_s":0.002368,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1350gitlab # [ 185.188781] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.759Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"ea0d5d0b314cd0f023abe7a1","created_at":"2026-09-01T10:34:37.308Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"89029b8de8a59834cc5045b8cb8c7c94","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.324Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-ea0d5d0b314cd0f023abe7a1: start","job_status":"start","queue_duration_s":3.434385,"scheduling_latency_s":3.434428}1351gitlab # [ 185.220822] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.820Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"ea0d5d0b314cd0f023abe7a1","created_at":"2026-09-01T10:34:37.308Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"89029b8de8a59834cc5045b8cb8c7c94","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-01T10:34:37.324Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-ea0d5d0b314cd0f023abe7a1: done: 0.061793 sec","job_status":"done","queue_duration_s":3.434385,"scheduling_latency_s":3.434428,"redis_calls":12,"redis_duration_s":0.004004,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000568,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001893,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001543,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030325,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001062984,"exclusive_lock_hold_duration_s":0.019495772999999997,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.061793,"completed_at":"2026-09-01T10:34:40.820Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020347,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1352gitlab # [ 185.303590] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.823Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"72956a8002fe266c7ede7b95","created_at":"2026-09-01T10:34:40.245Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-06d131133d6e4e58","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"118eae5e07a1eac8f7996b39cb453d1b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.260Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-72956a8002fe266c7ede7b95: start","job_status":"start","queue_duration_s":0.562788,"scheduling_latency_s":0.562822}1353gitlab # [ 185.305485] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.846Z","message":"No existing merge request to be cleaned up."}1354gitlab # [ 185.306244] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.866Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"72956a8002fe266c7ede7b95","created_at":"2026-09-01T10:34:40.245Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-06d131133d6e4e58","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"118eae5e07a1eac8f7996b39cb453d1b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.260Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-72956a8002fe266c7ede7b95: done: 0.042993 sec","job_status":"done","queue_duration_s":0.562788,"scheduling_latency_s":0.562822,"redis_calls":13,"redis_duration_s":0.004353,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000691,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002161,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02719,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000414857,"duration_s":0.042993,"completed_at":"2026-09-01T10:34:40.865Z","load_balancing_strategy":"primary","db_duration_s":0.004907,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1355gitlab # [ 185.371903] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.868Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bc2f3a30f087915e966a69bf","created_at":"2026-09-01T10:34:40.245Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-06d131133d6e4e58","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"118eae5e07a1eac8f7996b39cb453d1b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.260Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bc2f3a30f087915e966a69bf: start","job_status":"start","queue_duration_s":0.608014,"scheduling_latency_s":0.608046}1356gitlab # [ 185.373074] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.884Z","message":"No existing merge request to be cleaned up."}1357gitlab # [ 185.373771] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.896Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bc2f3a30f087915e966a69bf","created_at":"2026-09-01T10:34:40.245Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-06d131133d6e4e58","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"118eae5e07a1eac8f7996b39cb453d1b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.260Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bc2f3a30f087915e966a69bf: done: 0.028214 sec","job_status":"done","queue_duration_s":0.608014,"scheduling_latency_s":0.608046,"redis_calls":10,"redis_duration_s":0.003307,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001334,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001973,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028214,"completed_at":"2026-09-01T10:34:40.896Z","load_balancing_strategy":"primary","db_duration_s":0.002501,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1358gitlab # [ 185.468966] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.899Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c72c6e6fde12ea174e0f7236","created_at":"2026-09-01T10:34:40.245Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-06d131133d6e4e58","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"118eae5e07a1eac8f7996b39cb453d1b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.260Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c72c6e6fde12ea174e0f7236: start","job_status":"start","queue_duration_s":0.638722,"scheduling_latency_s":0.638754}1359gitlab # [ 185.471078] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.914Z","message":"No existing merge request to be cleaned up."}1360gitlab # [ 185.471688] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.923Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c72c6e6fde12ea174e0f7236","created_at":"2026-09-01T10:34:40.245Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-06d131133d6e4e58","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"118eae5e07a1eac8f7996b39cb453d1b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.260Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c72c6e6fde12ea174e0f7236: done: 0.024575 sec","job_status":"done","queue_duration_s":0.638722,"scheduling_latency_s":0.638754,"redis_calls":10,"redis_duration_s":0.002707,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001645,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016105,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024575,"completed_at":"2026-09-01T10:34:40.923Z","load_balancing_strategy":"primary","db_duration_s":0.002261,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1361gitlab # [ 185.539054] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.925Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f9cb52b87d43a05099a66840","created_at":"2026-09-01T10:34:40.245Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-06d131133d6e4e58","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"118eae5e07a1eac8f7996b39cb453d1b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.260Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f9cb52b87d43a05099a66840: start","job_status":"start","queue_duration_s":0.665021,"scheduling_latency_s":0.665049}1362gitlab # [ 185.540890] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.938Z","message":"No existing merge request to be cleaned up."}1363gitlab # [ 185.542953] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.948Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f9cb52b87d43a05099a66840","created_at":"2026-09-01T10:34:40.245Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-06d131133d6e4e58","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"118eae5e07a1eac8f7996b39cb453d1b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.260Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f9cb52b87d43a05099a66840: done: 0.023389 sec","job_status":"done","queue_duration_s":0.665021,"scheduling_latency_s":0.665049,"redis_calls":10,"redis_duration_s":0.0028929999999999997,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001157,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001736,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023389,"completed_at":"2026-09-01T10:34:40.948Z","load_balancing_strategy":"primary","db_duration_s":0.002033,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1364gitlab # [ 185.575883] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.950Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43175cdb1cef42fea5f2f05a","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43175cdb1cef42fea5f2f05a: start","job_status":"start","queue_duration_s":0.490391,"scheduling_latency_s":0.490417}1365gitlab # [ 185.640414] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43175cdb1cef42fea5f2f05a","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43175cdb1cef42fea5f2f05a: done: 0.02895 sec","job_status":"done","queue_duration_s":0.490391,"scheduling_latency_s":0.490417,"redis_calls":16,"redis_duration_s":0.004494,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000554,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002384,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000402007,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02895,"completed_at":"2026-09-01T10:34:40.979Z","load_balancing_strategy":"primary","db_duration_s":0.00073,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1366gitlab # [ 185.708980] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:40.981Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ead22580413fea4baf2a916e","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ead22580413fea4baf2a916e: start","job_status":"start","queue_duration_s":0.52139,"scheduling_latency_s":0.521415}1367gitlab # [ 185.710134] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.000Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ead22580413fea4baf2a916e","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ead22580413fea4baf2a916e: done: 0.018559 sec","job_status":"done","queue_duration_s":0.52139,"scheduling_latency_s":0.521415,"redis_calls":13,"redis_duration_s":0.00304,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000835,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002205,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018559,"completed_at":"2026-09-01T10:34:41.000Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1368gitlab # [ 185.725552] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.001Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d71ace2bfb972e4ce5c244b","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d71ace2bfb972e4ce5c244b: start","job_status":"start","queue_duration_s":0.541426,"scheduling_latency_s":0.541488}1369gitlab # [ 185.792637] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.020Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d71ace2bfb972e4ce5c244b","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d71ace2bfb972e4ce5c244b: done: 0.019169 sec","job_status":"done","queue_duration_s":0.541426,"scheduling_latency_s":0.541488,"redis_calls":13,"redis_duration_s":0.002876,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000841,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002035,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019169,"completed_at":"2026-09-01T10:34:41.020Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1370gitlab # [ 185.797458] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.022Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2d24fbd6e0697500361834d","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2d24fbd6e0697500361834d: start","job_status":"start","queue_duration_s":0.562133,"scheduling_latency_s":0.56216}1371gitlab # [ 185.814228] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.042Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2d24fbd6e0697500361834d","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2d24fbd6e0697500361834d: done: 0.020023 sec","job_status":"done","queue_duration_s":0.562133,"scheduling_latency_s":0.56216,"redis_calls":13,"redis_duration_s":0.002998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000985,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012979,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020023,"completed_at":"2026-09-01T10:34:41.042Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1372gitlab # [ 185.878385] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.044Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ec7e73a4658c27a85dc50f3","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ec7e73a4658c27a85dc50f3: start","job_status":"start","queue_duration_s":0.584049,"scheduling_latency_s":0.584085}1373gitlab # [ 185.884396] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.070Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ec7e73a4658c27a85dc50f3","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ec7e73a4658c27a85dc50f3: done: 0.026572 sec","job_status":"done","queue_duration_s":0.584049,"scheduling_latency_s":0.584085,"redis_calls":13,"redis_duration_s":0.004669,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001221,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003448,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017401,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026572,"completed_at":"2026-09-01T10:34:41.070Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1374gitlab # [ 185.955049] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.073Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb719bbe06af010ed278d0da","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb719bbe06af010ed278d0da: start","job_status":"start","queue_duration_s":0.61304,"scheduling_latency_s":0.613078}1375gitlab # [ 185.956342] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.095Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb719bbe06af010ed278d0da","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb719bbe06af010ed278d0da: done: 0.022438 sec","job_status":"done","queue_duration_s":0.61304,"scheduling_latency_s":0.613078,"redis_calls":13,"redis_duration_s":0.003483,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001074,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002409,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014803,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022438,"completed_at":"2026-09-01T10:34:41.095Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1376gitlab # [ 186.031284] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.097Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76913547e8ee5cad8e49a5c7","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76913547e8ee5cad8e49a5c7: start","job_status":"start","queue_duration_s":0.637332,"scheduling_latency_s":0.63736}1377gitlab # [ 186.032432] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.123Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76913547e8ee5cad8e49a5c7","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76913547e8ee5cad8e49a5c7: done: 0.025414 sec","job_status":"done","queue_duration_s":0.637332,"scheduling_latency_s":0.63736,"redis_calls":13,"redis_duration_s":0.005424,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001422,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004002,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016796,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025414,"completed_at":"2026-09-01T10:34:41.122Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1378gitlab # [ 186.095859] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3c7229096a9fc13a59f16a1","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3c7229096a9fc13a59f16a1: start","job_status":"start","queue_duration_s":0.665112,"scheduling_latency_s":0.665152}1379gitlab # [ 186.102755] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.155Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3c7229096a9fc13a59f16a1","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3c7229096a9fc13a59f16a1: done: 0.029976 sec","job_status":"done","queue_duration_s":0.665112,"scheduling_latency_s":0.665152,"redis_calls":13,"redis_duration_s":0.00676,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002236,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004524,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029976,"completed_at":"2026-09-01T10:34:41.155Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1380gitlab # [ 186.206296] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.159Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c366bfd8759b194ee47b401","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c366bfd8759b194ee47b401: start","job_status":"start","queue_duration_s":0.699034,"scheduling_latency_s":0.699072}1381gitlab # [ 186.207419] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.192Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c366bfd8759b194ee47b401","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c366bfd8759b194ee47b401: done: 0.032752 sec","job_status":"done","queue_duration_s":0.699034,"scheduling_latency_s":0.699072,"redis_calls":13,"redis_duration_s":0.008345,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003467,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004878,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032752,"completed_at":"2026-09-01T10:34:41.191Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1382gitlab # [ 186.278443] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.195Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a7e8fd1a8f1375de12c0fee","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a7e8fd1a8f1375de12c0fee: start","job_status":"start","queue_duration_s":0.734649,"scheduling_latency_s":0.734703}1383gitlab # [ 186.281153] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a7e8fd1a8f1375de12c0fee","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a7e8fd1a8f1375de12c0fee: done: 0.025591 sec","job_status":"done","queue_duration_s":0.734649,"scheduling_latency_s":0.734703,"redis_calls":13,"redis_duration_s":0.004158,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001255,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002903,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015911,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025591,"completed_at":"2026-09-01T10:34:41.220Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1384gitlab # [ 186.349527] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.222Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ba3f3b6dc79256994ca6d2b","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ba3f3b6dc79256994ca6d2b: start","job_status":"start","queue_duration_s":0.761952,"scheduling_latency_s":0.761981}1385gitlab # [ 186.356291] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.245Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ba3f3b6dc79256994ca6d2b","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ba3f3b6dc79256994ca6d2b: done: 0.023022 sec","job_status":"done","queue_duration_s":0.761952,"scheduling_latency_s":0.761981,"redis_calls":13,"redis_duration_s":0.004021,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002483,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023022,"completed_at":"2026-09-01T10:34:41.245Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1386gitlab # [ 186.431172] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.248Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18efe8efb1011aaa11bfd08e","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18efe8efb1011aaa11bfd08e: start","job_status":"start","queue_duration_s":0.787995,"scheduling_latency_s":0.788026}1387gitlab # [ 186.432770] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.272Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18efe8efb1011aaa11bfd08e","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18efe8efb1011aaa11bfd08e: done: 0.02411 sec","job_status":"done","queue_duration_s":0.787995,"scheduling_latency_s":0.788026,"redis_calls":13,"redis_duration_s":0.003824,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001278,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002546,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015138,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02411,"completed_at":"2026-09-01T10:34:41.272Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1388gitlab # [ 186.525603] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.274Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6a1b51220821a6356f4602c","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6a1b51220821a6356f4602c: start","job_status":"start","queue_duration_s":0.814196,"scheduling_latency_s":0.814228}1389gitlab # [ 186.527295] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.300Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6a1b51220821a6356f4602c","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6a1b51220821a6356f4602c: done: 0.026445 sec","job_status":"done","queue_duration_s":0.814196,"scheduling_latency_s":0.814228,"redis_calls":13,"redis_duration_s":0.005012,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001166,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003846,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016226,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026445,"completed_at":"2026-09-01T10:34:41.300Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1390gitlab # [ 186.598815] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.303Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e77512bcb109c8201f1b9a1","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e77512bcb109c8201f1b9a1: start","job_status":"start","queue_duration_s":0.842776,"scheduling_latency_s":0.842809}1391gitlab # [ 186.600623] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.342Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e77512bcb109c8201f1b9a1","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e77512bcb109c8201f1b9a1: done: 0.039685 sec","job_status":"done","queue_duration_s":0.842776,"scheduling_latency_s":0.842809,"redis_calls":13,"redis_duration_s":0.013214,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003171,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.010043,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020302,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.039685,"completed_at":"2026-09-01T10:34:41.342Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1392gitlab # [ 186.633432] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.345Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d03787c2ae176a9afe42dbcb","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d03787c2ae176a9afe42dbcb: start","job_status":"start","queue_duration_s":0.885408,"scheduling_latency_s":0.885442}1393gitlab # [ 186.652073] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.372Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d03787c2ae176a9afe42dbcb","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d03787c2ae176a9afe42dbcb: done: 0.026813 sec","job_status":"done","queue_duration_s":0.885408,"scheduling_latency_s":0.885442,"redis_calls":13,"redis_duration_s":0.0059770000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001184,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004793,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015763,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026813,"completed_at":"2026-09-01T10:34:41.372Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1394gitlab # [ 186.750361] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.375Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64fd81596328cd617967ecb8","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64fd81596328cd617967ecb8: start","job_status":"start","queue_duration_s":0.91515,"scheduling_latency_s":0.915189}1395gitlab # [ 186.751821] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.403Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64fd81596328cd617967ecb8","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64fd81596328cd617967ecb8: done: 0.027523 sec","job_status":"done","queue_duration_s":0.91515,"scheduling_latency_s":0.915189,"redis_calls":13,"redis_duration_s":0.005596,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001479,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004117,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015492,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027523,"completed_at":"2026-09-01T10:34:41.402Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1396gitlab # [ 186.830125] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.407Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a864c423dc31aeaa6f5af400","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a864c423dc31aeaa6f5af400: start","job_status":"start","queue_duration_s":0.946978,"scheduling_latency_s":0.947019}1397gitlab # [ 186.831176] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.458Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a864c423dc31aeaa6f5af400","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a864c423dc31aeaa6f5af400: done: 0.051008 sec","job_status":"done","queue_duration_s":0.946978,"scheduling_latency_s":0.947019,"redis_calls":13,"redis_duration_s":0.025096999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.022451,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002646,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018229,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.051008,"completed_at":"2026-09-01T10:34:41.458Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1398gitlab # [ 186.855189] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.461Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"527aabc808f0341e80e7ba93","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-527aabc808f0341e80e7ba93: start","job_status":"start","queue_duration_s":1.000691,"scheduling_latency_s":1.00073}1399gitlab # [ 186.872075] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"527aabc808f0341e80e7ba93","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-527aabc808f0341e80e7ba93: done: 0.02937 sec","job_status":"done","queue_duration_s":1.000691,"scheduling_latency_s":1.00073,"redis_calls":13,"redis_duration_s":0.003973,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001357,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002616,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018228,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02937,"completed_at":"2026-09-01T10:34:41.490Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1400gitlab # [ 186.935248] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.492Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a59c083612018709263a98a7","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a59c083612018709263a98a7: start","job_status":"start","queue_duration_s":1.032358,"scheduling_latency_s":1.032391}1401gitlab # [ 186.942324] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.621Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a59c083612018709263a98a7","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a59c083612018709263a98a7: done: 0.128754 sec","job_status":"done","queue_duration_s":1.032358,"scheduling_latency_s":1.032391,"redis_calls":13,"redis_duration_s":0.004275,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001612,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002663,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.128754,"completed_at":"2026-09-01T10:34:41.621Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1402gitlab # [ 187.006247] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.623Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"367aed00676b6102e419f06c","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-367aed00676b6102e419f06c: start","job_status":"start","queue_duration_s":1.163521,"scheduling_latency_s":1.163557}1403gitlab # [ 187.065625] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"367aed00676b6102e419f06c","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-367aed00676b6102e419f06c: done: 0.02691 sec","job_status":"done","queue_duration_s":1.163521,"scheduling_latency_s":1.163557,"redis_calls":13,"redis_duration_s":0.003736,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001216,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00252,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02691,"completed_at":"2026-09-01T10:34:41.650Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1404gitlab # [ 187.067258] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.652Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f834de041687ad03f74b1bff","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f834de041687ad03f74b1bff: start","job_status":"start","queue_duration_s":1.192033,"scheduling_latency_s":1.192058}1405gitlab # [ 187.130265] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.675Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f834de041687ad03f74b1bff","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f834de041687ad03f74b1bff: done: 0.023643 sec","job_status":"done","queue_duration_s":1.192033,"scheduling_latency_s":1.192058,"redis_calls":13,"redis_duration_s":0.003943,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001296,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002647,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015176,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023643,"completed_at":"2026-09-01T10:34:41.675Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1406gitlab # [ 187.140268] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6747069edaa66a59e6432335","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6747069edaa66a59e6432335: start","job_status":"start","queue_duration_s":1.219817,"scheduling_latency_s":1.219861}1407gitlab # [ 187.207189] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6747069edaa66a59e6432335","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6747069edaa66a59e6432335: done: 0.036115 sec","job_status":"done","queue_duration_s":1.219817,"scheduling_latency_s":1.219861,"redis_calls":13,"redis_duration_s":0.013076,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.010527,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018812,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.036115,"completed_at":"2026-09-01T10:34:41.716Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1408gitlab # [ 187.209392] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.719Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f2e1cc45f476d9bd9b9ec2b","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f2e1cc45f476d9bd9b9ec2b: start","job_status":"start","queue_duration_s":1.258971,"scheduling_latency_s":1.259027}1409gitlab # [ 187.226790] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f2e1cc45f476d9bd9b9ec2b","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f2e1cc45f476d9bd9b9ec2b: done: 0.03845 sec","job_status":"done","queue_duration_s":1.258971,"scheduling_latency_s":1.259027,"redis_calls":13,"redis_duration_s":0.011504,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002689,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008815,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019193,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03845,"completed_at":"2026-09-01T10:34:41.757Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410gitlab # [ 187.279258] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.760Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d524a10b7710ff8ed566c2f0","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d524a10b7710ff8ed566c2f0: start","job_status":"start","queue_duration_s":1.300173,"scheduling_latency_s":1.300223}1411gitlab # [ 187.341950] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.787Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d524a10b7710ff8ed566c2f0","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d524a10b7710ff8ed566c2f0: done: 0.026791 sec","job_status":"done","queue_duration_s":1.300173,"scheduling_latency_s":1.300223,"redis_calls":13,"redis_duration_s":0.00561,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002022,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003588,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016421,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026791,"completed_at":"2026-09-01T10:34:41.787Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412gitlab # [ 187.344227] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62d25e7b536d6095c6a4c09b","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62d25e7b536d6095c6a4c09b: start","job_status":"start","queue_duration_s":1.329375,"scheduling_latency_s":1.329413}1413gitlab # [ 187.362590] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.824Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62d25e7b536d6095c6a4c09b","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62d25e7b536d6095c6a4c09b: done: 0.034439 sec","job_status":"done","queue_duration_s":1.329375,"scheduling_latency_s":1.329413,"redis_calls":13,"redis_duration_s":0.010469,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002161,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008308,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018191,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034439,"completed_at":"2026-09-01T10:34:41.824Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1414gitlab # [ 187.441870] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ecfd8fe096bfa18a41a3211","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ecfd8fe096bfa18a41a3211: start","job_status":"start","queue_duration_s":1.366172,"scheduling_latency_s":1.366221}1415gitlab # [ 187.443065] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.852Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ecfd8fe096bfa18a41a3211","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ecfd8fe096bfa18a41a3211: done: 0.026052 sec","job_status":"done","queue_duration_s":1.366172,"scheduling_latency_s":1.366221,"redis_calls":13,"redis_duration_s":0.004863,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00173,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003133,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016239,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026052,"completed_at":"2026-09-01T10:34:41.852Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416gitlab # [ 187.504395] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.855Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"116e22cab0c2cd1e0f12c548","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-116e22cab0c2cd1e0f12c548: start","job_status":"start","queue_duration_s":1.394894,"scheduling_latency_s":1.394934}1417gitlab # [ 187.505915] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"116e22cab0c2cd1e0f12c548","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-116e22cab0c2cd1e0f12c548: done: 0.03494 sec","job_status":"done","queue_duration_s":1.394894,"scheduling_latency_s":1.394934,"redis_calls":13,"redis_duration_s":0.007582,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004323,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003259,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03494,"completed_at":"2026-09-01T10:34:41.890Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1418gitlab # [ 187.585320] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.893Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6be3341b4a79fa0fa4163674","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6be3341b4a79fa0fa4163674: start","job_status":"start","queue_duration_s":1.433139,"scheduling_latency_s":1.433196}1419gitlab # [ 187.586323] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6be3341b4a79fa0fa4163674","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6be3341b4a79fa0fa4163674: done: 0.03046 sec","job_status":"done","queue_duration_s":1.433139,"scheduling_latency_s":1.433196,"redis_calls":13,"redis_duration_s":0.004757,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001865,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002892,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03046,"completed_at":"2026-09-01T10:34:41.923Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1420gitlab # [ 187.666620] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:41.927Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9018c08956d438fa0e000a2","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9018c08956d438fa0e000a2: start","job_status":"start","queue_duration_s":1.466688,"scheduling_latency_s":1.466729}1421gitlab # [ 187.667481] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.023Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9018c08956d438fa0e000a2","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9018c08956d438fa0e000a2: done: 0.096992 sec","job_status":"done","queue_duration_s":1.466688,"scheduling_latency_s":1.466729,"redis_calls":13,"redis_duration_s":0.006335,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002387,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003948,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079713,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.096992,"completed_at":"2026-09-01T10:34:42.023Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1422gitlab # [ 187.748545] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.026Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29e8c2946d57a600775be8ab","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29e8c2946d57a600775be8ab: start","job_status":"start","queue_duration_s":1.565851,"scheduling_latency_s":1.565885}1423gitlab # [ 187.750275] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.055Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29e8c2946d57a600775be8ab","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29e8c2946d57a600775be8ab: done: 0.029376 sec","job_status":"done","queue_duration_s":1.565851,"scheduling_latency_s":1.565885,"redis_calls":13,"redis_duration_s":0.009358,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002413,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015637,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029376,"completed_at":"2026-09-01T10:34:42.055Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1424gitlab # [ 187.770591] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.058Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf13354e9b85c3ea4aa34163","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf13354e9b85c3ea4aa34163: start","job_status":"start","queue_duration_s":1.598176,"scheduling_latency_s":1.598224}1425gitlab # [ 187.838480] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.080Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf13354e9b85c3ea4aa34163","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf13354e9b85c3ea4aa34163: done: 0.021624 sec","job_status":"done","queue_duration_s":1.598176,"scheduling_latency_s":1.598224,"redis_calls":13,"redis_duration_s":0.003438,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001059,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002379,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021624,"completed_at":"2026-09-01T10:34:42.080Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1426gitlab # [ 187.846098] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.081Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91898785ae6cf59a34985e2b","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91898785ae6cf59a34985e2b: start","job_status":"start","queue_duration_s":1.62158,"scheduling_latency_s":1.62161}1427gitlab # [ 187.916914] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.102Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91898785ae6cf59a34985e2b","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91898785ae6cf59a34985e2b: done: 0.020229 sec","job_status":"done","queue_duration_s":1.62158,"scheduling_latency_s":1.62161,"redis_calls":13,"redis_duration_s":0.0031520000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001004,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002148,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020229,"completed_at":"2026-09-01T10:34:42.101Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1428gitlab # [ 187.918393] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.103Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"511b95b96ea0b0323610ca0e","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-511b95b96ea0b0323610ca0e: start","job_status":"start","queue_duration_s":1.643429,"scheduling_latency_s":1.643499}1429gitlab # [ 187.981404] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"511b95b96ea0b0323610ca0e","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-511b95b96ea0b0323610ca0e: done: 0.021326 sec","job_status":"done","queue_duration_s":1.643429,"scheduling_latency_s":1.643499,"redis_calls":13,"redis_duration_s":0.003704,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00154,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002164,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021326,"completed_at":"2026-09-01T10:34:42.125Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1430gitlab # [ 187.984082] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.130Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7eeac00b766d69b35d94ed0d","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7eeac00b766d69b35d94ed0d: start","job_status":"start","queue_duration_s":1.670088,"scheduling_latency_s":1.670127}1431gitlab # [ 188.002572] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7eeac00b766d69b35d94ed0d","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7eeac00b766d69b35d94ed0d: done: 0.021818 sec","job_status":"done","queue_duration_s":1.670088,"scheduling_latency_s":1.670127,"redis_calls":13,"redis_duration_s":0.0034210000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001061,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00236,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021818,"completed_at":"2026-09-01T10:34:42.152Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1432gitlab # [ 188.068098] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.154Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84c8e0bb3b70a9ce01ee6bde","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84c8e0bb3b70a9ce01ee6bde: start","job_status":"start","queue_duration_s":1.694395,"scheduling_latency_s":1.694424}1433gitlab # [ 188.132642] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.181Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84c8e0bb3b70a9ce01ee6bde","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84c8e0bb3b70a9ce01ee6bde: done: 0.027084 sec","job_status":"done","queue_duration_s":1.694395,"scheduling_latency_s":1.694424,"redis_calls":13,"redis_duration_s":0.006116,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001162,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004954,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016764,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027084,"completed_at":"2026-09-01T10:34:42.181Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1434gitlab # [ 188.134224] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.184Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f82da20f22bf2a6a849b0183","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f82da20f22bf2a6a849b0183: start","job_status":"start","queue_duration_s":1.72409,"scheduling_latency_s":1.724124}1435gitlab # [ 188.207231] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.221Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f82da20f22bf2a6a849b0183","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f82da20f22bf2a6a849b0183: done: 0.037216 sec","job_status":"done","queue_duration_s":1.72409,"scheduling_latency_s":1.724124,"redis_calls":13,"redis_duration_s":0.011606,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002936,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00867,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019158,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.037216,"completed_at":"2026-09-01T10:34:42.221Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1436gitlab # [ 188.208862] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.225Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6142afc3bd9b4a6d3937633","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6142afc3bd9b4a6d3937633: start","job_status":"start","queue_duration_s":1.764943,"scheduling_latency_s":1.765019}1437gitlab # [ 188.219584] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.258Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6142afc3bd9b4a6d3937633","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6142afc3bd9b4a6d3937633: done: 0.032954 sec","job_status":"done","queue_duration_s":1.764943,"scheduling_latency_s":1.765019,"redis_calls":13,"redis_duration_s":0.011068,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007575,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016429,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032954,"completed_at":"2026-09-01T10:34:42.258Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1438gitlab # [ 188.283782] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.261Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b807e5c8abaa2d1e89a1dad","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b807e5c8abaa2d1e89a1dad: start","job_status":"start","queue_duration_s":1.800829,"scheduling_latency_s":1.80086}1439gitlab # [ 188.288311] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.284Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b807e5c8abaa2d1e89a1dad","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b807e5c8abaa2d1e89a1dad: done: 0.023343 sec","job_status":"done","queue_duration_s":1.800829,"scheduling_latency_s":1.80086,"redis_calls":13,"redis_duration_s":0.0037010000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001583,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002118,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023343,"completed_at":"2026-09-01T10:34:42.284Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1440gitlab # [ 188.376304] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.286Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a9ecb586a43868c4e53d6c7","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a9ecb586a43868c4e53d6c7: start","job_status":"start","queue_duration_s":1.826022,"scheduling_latency_s":1.82605}1441gitlab # [ 188.377800] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.305Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a9ecb586a43868c4e53d6c7","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a9ecb586a43868c4e53d6c7: done: 0.01871 sec","job_status":"done","queue_duration_s":1.826022,"scheduling_latency_s":1.82605,"redis_calls":13,"redis_duration_s":0.002855,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01871,"completed_at":"2026-09-01T10:34:42.304Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1442gitlab # [ 188.454219] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.306Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de494b0c5b3b1dc93b8bee17","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de494b0c5b3b1dc93b8bee17: start","job_status":"start","queue_duration_s":1.846207,"scheduling_latency_s":1.846232}1443gitlab # [ 188.455295] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de494b0c5b3b1dc93b8bee17","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de494b0c5b3b1dc93b8bee17: done: 0.025634 sec","job_status":"done","queue_duration_s":1.846207,"scheduling_latency_s":1.846232,"redis_calls":13,"redis_duration_s":0.003661,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001152,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002509,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018333,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025634,"completed_at":"2026-09-01T10:34:42.332Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1444gitlab # [ 188.525903] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.334Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"80ddbeb9483edae826bf9396","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-80ddbeb9483edae826bf9396: start","job_status":"start","queue_duration_s":1.874141,"scheduling_latency_s":1.874177}1445gitlab # [ 188.526987] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.363Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"80ddbeb9483edae826bf9396","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-80ddbeb9483edae826bf9396: done: 0.029345 sec","job_status":"done","queue_duration_s":1.874141,"scheduling_latency_s":1.874177,"redis_calls":13,"redis_duration_s":0.003633,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001148,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002485,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017869,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029345,"completed_at":"2026-09-01T10:34:42.363Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1446gitlab # [ 188.594139] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.367Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b94f3ad806262c8d6c0c1415","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b94f3ad806262c8d6c0c1415: start","job_status":"start","queue_duration_s":1.906852,"scheduling_latency_s":1.906887}1447gitlab # [ 188.595331] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.395Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b94f3ad806262c8d6c0c1415","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b94f3ad806262c8d6c0c1415: done: 0.028468 sec","job_status":"done","queue_duration_s":1.906852,"scheduling_latency_s":1.906887,"redis_calls":13,"redis_duration_s":0.003837,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001211,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002626,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028468,"completed_at":"2026-09-01T10:34:42.395Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1448gitlab # [ 188.680666] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e748c974d31c60aecb4ea331","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e748c974d31c60aecb4ea331: start","job_status":"start","queue_duration_s":1.937599,"scheduling_latency_s":1.937631}1449gitlab # [ 188.682544] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.418Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e748c974d31c60aecb4ea331","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e748c974d31c60aecb4ea331: done: 0.020974 sec","job_status":"done","queue_duration_s":1.937599,"scheduling_latency_s":1.937631,"redis_calls":13,"redis_duration_s":0.003105,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001056,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002049,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013248,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020974,"completed_at":"2026-09-01T10:34:42.418Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1450gitlab # [ 188.756539] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.420Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7df3c7671186751908448756","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7df3c7671186751908448756: start","job_status":"start","queue_duration_s":1.960097,"scheduling_latency_s":1.960122}1451gitlab # [ 188.757987] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.439Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7df3c7671186751908448756","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7df3c7671186751908448756: done: 0.019026 sec","job_status":"done","queue_duration_s":1.960097,"scheduling_latency_s":1.960122,"redis_calls":13,"redis_duration_s":0.002973,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000965,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002008,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012341,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019026,"completed_at":"2026-09-01T10:34:42.439Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1452gitlab # [ 188.770429] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6bdbc11b0aff9a0adbb653f6","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6bdbc11b0aff9a0adbb653f6: start","job_status":"start","queue_duration_s":1.980888,"scheduling_latency_s":1.980915}1453gitlab # [ 188.833258] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.462Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6bdbc11b0aff9a0adbb653f6","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6bdbc11b0aff9a0adbb653f6: done: 0.020818 sec","job_status":"done","queue_duration_s":1.980888,"scheduling_latency_s":1.980915,"redis_calls":13,"redis_duration_s":0.003105,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000929,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002176,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013196,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020818,"completed_at":"2026-09-01T10:34:42.461Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1454gitlab # [ 188.838914] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.463Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e367a5ca3684c2bc7a2ec9ef","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e367a5ca3684c2bc7a2ec9ef: start","job_status":"start","queue_duration_s":2.003493,"scheduling_latency_s":2.003523}1455gitlab # [ 188.856097] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.484Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e367a5ca3684c2bc7a2ec9ef","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e367a5ca3684c2bc7a2ec9ef: done: 0.020882 sec","job_status":"done","queue_duration_s":2.003493,"scheduling_latency_s":2.003523,"redis_calls":13,"redis_duration_s":0.003209,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001048,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002161,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013615,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020882,"completed_at":"2026-09-01T10:34:42.484Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1456gitlab # [ 188.940495] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.486Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4cf4e51c94ad5220962d76b0","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4cf4e51c94ad5220962d76b0: start","job_status":"start","queue_duration_s":2.026374,"scheduling_latency_s":2.026407}1457gitlab # [ 188.941857] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.508Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4cf4e51c94ad5220962d76b0","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4cf4e51c94ad5220962d76b0: done: 0.021842 sec","job_status":"done","queue_duration_s":2.026374,"scheduling_latency_s":2.026407,"redis_calls":13,"redis_duration_s":0.003536,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00112,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002416,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014684,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021842,"completed_at":"2026-09-01T10:34:42.508Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1458gitlab # [ 189.004830] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.510Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bfc2558ae8f170463bdf1004","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bfc2558ae8f170463bdf1004: start","job_status":"start","queue_duration_s":2.050442,"scheduling_latency_s":2.050516}1459gitlab # [ 189.006036] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.532Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bfc2558ae8f170463bdf1004","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bfc2558ae8f170463bdf1004: done: 0.022198 sec","job_status":"done","queue_duration_s":2.050442,"scheduling_latency_s":2.050516,"redis_calls":13,"redis_duration_s":0.003366,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001102,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002264,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013819,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022198,"completed_at":"2026-09-01T10:34:42.532Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1460gitlab # [ 189.068983] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.534Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29ff369e473b5270633eb9a0","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29ff369e473b5270633eb9a0: start","job_status":"start","queue_duration_s":2.074248,"scheduling_latency_s":2.074273}1461gitlab # [ 189.070156] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.555Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29ff369e473b5270633eb9a0","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29ff369e473b5270633eb9a0: done: 0.020875 sec","job_status":"done","queue_duration_s":2.074248,"scheduling_latency_s":2.074273,"redis_calls":13,"redis_duration_s":0.003375,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001212,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002163,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020875,"completed_at":"2026-09-01T10:34:42.555Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1462gitlab # [ 189.136265] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.556Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc6252b019422aac688a8bdd","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc6252b019422aac688a8bdd: start","job_status":"start","queue_duration_s":2.096663,"scheduling_latency_s":2.096687}1463gitlab # [ 189.137357] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:34:42.577Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc6252b019422aac688a8bdd","created_at":"2026-09-01T10:34:40.371Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-a2a37634137a4ee8","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"647683ff4b12ae24356015ff98a49b1d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:34:40.460Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc6252b019422aac688a8bdd: done: 0.02095 sec","job_status":"done","queue_duration_s":2.096663,"scheduling_latency_s":2.096687,"redis_calls":13,"redis_duration_s":0.003175,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002236,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014046,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02095,"completed_at":"2026-09-01T10:34:42.577Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1464gitlab # [ 189.499399] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 2964920163 wd_nsec: 29649189361465gitlab # [ 210.494497] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:06.093Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"f7457a5dde93fc49d9d02d85","created_at":"2026-09-01T10:35:06.032Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2a3ae172fe61eb3da52da243e88b0e23","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.083Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-f7457a5dde93fc49d9d02d85: start","job_status":"start","queue_duration_s":0.009936,"scheduling_latency_s":0.009986}1466gitlab # [ 210.585428] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:06.185Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"f7457a5dde93fc49d9d02d85","created_at":"2026-09-01T10:35:06.032Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2a3ae172fe61eb3da52da243e88b0e23","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.083Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-f7457a5dde93fc49d9d02d85: done: 0.09165 sec","job_status":"done","queue_duration_s":0.009936,"scheduling_latency_s":0.009986,"redis_calls":7,"redis_duration_s":0.018201000000000002,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.011559,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.006642,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018171,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.09165,"completed_at":"2026-09-01T10:35:06.185Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1467gitlab # [ 210.656158] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:06.254Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"49b9ac6e6637495108647672","created_at":"2026-09-01T10:35:06.130Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"77d4aa0f5abe66172aac65d08b009f90","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.163Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-49b9ac6e6637495108647672: start","job_status":"start","queue_duration_s":0.090865,"scheduling_latency_s":0.090913}1468gitlab # [ 211.148442] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:06.748Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"49b9ac6e6637495108647672","created_at":"2026-09-01T10:35:06.130Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"77d4aa0f5abe66172aac65d08b009f90","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.163Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-49b9ac6e6637495108647672: done: 0.494002 sec","job_status":"done","queue_duration_s":0.090865,"scheduling_latency_s":0.090913,"redis_calls":12,"redis_duration_s":0.036406999999999995,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.01587,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.011936,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.008601,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.04,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.034,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.270824,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.005322185,"exclusive_lock_hold_duration_s":0.353499398,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.494002,"completed_at":"2026-09-01T10:35:06.747Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.081021,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null}1469gitlab # [ 211.220986] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:06.755Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"c72c7c3a895f88003bb24445","created_at":"2026-09-01T10:35:06.250Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5f66874619851e5eb62148c94140f26d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.261Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-c72c7c3a895f88003bb24445: start","job_status":"start","queue_duration_s":0.494119,"scheduling_latency_s":0.494176}1470gitlab # [ 211.395901] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:06.995Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1471gitlab # [ 211.434169] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:07.029Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"c72c7c3a895f88003bb24445","created_at":"2026-09-01T10:35:06.250Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5f66874619851e5eb62148c94140f26d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.261Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-c72c7c3a895f88003bb24445: done: 0.27112 sec","job_status":"done","queue_duration_s":0.494119,"scheduling_latency_s":0.494176,"redis_calls":7,"redis_duration_s":0.195824,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.180644,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.01518,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.27112,"completed_at":"2026-09-01T10:35:07.026Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1472gitlab # [ 211.576560] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:07.175Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"cea9bb9173e52cd032a41766","created_at":"2026-09-01T10:35:06.300Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1802dd6e97734ae95a1ff4b6700f979a","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.311Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-cea9bb9173e52cd032a41766: start","job_status":"start","queue_duration_s":0.863606,"scheduling_latency_s":0.863665}1473gitlab # [ 211.900635] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:07.500Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"cea9bb9173e52cd032a41766","created_at":"2026-09-01T10:35:06.300Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1802dd6e97734ae95a1ff4b6700f979a","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.311Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-cea9bb9173e52cd032a41766: done: 0.325413 sec","job_status":"done","queue_duration_s":0.863606,"scheduling_latency_s":0.863665,"redis_calls":10,"redis_duration_s":0.057253,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.048047,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00325,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005956,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.114,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.114,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000612648,"duration_s":0.325413,"completed_at":"2026-09-01T10:35:07.500Z","load_balancing_strategy":"primary","db_duration_s":0.174025,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1474gitlab # [ 212.107563] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:07.511Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"e9da9e2ca45e777b036dce1e","created_at":"2026-09-01T10:35:06.334Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ca702ed38153d433c6cc3dae55c480af","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-01T10:35:06.340Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-e9da9e2ca45e777b036dce1e: start","job_status":"start","queue_duration_s":1.170688,"scheduling_latency_s":1.170732}1475gitlab # [ 212.142225] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:07.742Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"e9da9e2ca45e777b036dce1e","created_at":"2026-09-01T10:35:06.334Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ca702ed38153d433c6cc3dae55c480af","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-01T10:35:06.340Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-e9da9e2ca45e777b036dce1e: done: 0.230919 sec","job_status":"done","queue_duration_s":1.170688,"scheduling_latency_s":1.170732,"redis_calls":7,"redis_duration_s":0.070013,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0665,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003513,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03281,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.230919,"completed_at":"2026-09-01T10:35:07.742Z","load_balancing_strategy":"primary","db_duration_s":0.010192,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1476gitlab # [ 212.298881] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:07.758Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"cf4f8d7f4e525a429042c79b","created_at":"2026-09-01T10:35:06.365Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ce9ec07b0282b311ac03c637e68d42c","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.372Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-cf4f8d7f4e525a429042c79b: start","job_status":"start","queue_duration_s":1.38522,"scheduling_latency_s":1.38526}1477gitlab # [ 212.369145] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:07.804Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"cf4f8d7f4e525a429042c79b","created_at":"2026-09-01T10:35:06.365Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ce9ec07b0282b311ac03c637e68d42c","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.372Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-cf4f8d7f4e525a429042c79b: done: 0.045968 sec","job_status":"done","queue_duration_s":1.38522,"scheduling_latency_s":1.38526,"redis_calls":9,"redis_duration_s":0.003652,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002134,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001518,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01977,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00101214,"exclusive_lock_hold_duration_s":0.007711316,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.045968,"completed_at":"2026-09-01T10:35:07.803Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004871,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1478gitlab # [ 212.557714] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:07.808Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"59a3b0951c0265246e649b1f","created_at":"2026-09-01T10:35:06.424Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1255a6cf37985c0bd40da43387f64e60","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.431Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-59a3b0951c0265246e649b1f: start","job_status":"start","queue_duration_s":1.37705,"scheduling_latency_s":1.377092}1479gitlab # [ 214.062495] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:09.655Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"59a3b0951c0265246e649b1f","created_at":"2026-09-01T10:35:06.424Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1255a6cf37985c0bd40da43387f64e60","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.431Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-59a3b0951c0265246e649b1f: done: 1.84702 sec","job_status":"done","queue_duration_s":1.37705,"scheduling_latency_s":1.377092,"redis_calls":562,"redis_duration_s":1.044101,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.015523,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":1.028578,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.664963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":1.84702,"completed_at":"2026-09-01T10:35:09.655Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1480gitlab # [ 214.235058] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:09.826Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"306c51957127c712fa43622b","created_at":"2026-09-01T10:35:06.728Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e7eac26097441dd01c07f5898461fd63","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.745Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-306c51957127c712fa43622b: start","job_status":"start","queue_duration_s":3.081177,"scheduling_latency_s":3.081298}1481gitlab # [ 214.424582] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:10.021Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"306c51957127c712fa43622b","created_at":"2026-09-01T10:35:06.728Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e7eac26097441dd01c07f5898461fd63","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.745Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-306c51957127c712fa43622b: done: 0.35886 sec","job_status":"done","queue_duration_s":3.081177,"scheduling_latency_s":3.081298,"redis_calls":7,"redis_duration_s":0.084686,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.076268,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.008418,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042704,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.35886,"completed_at":"2026-09-01T10:35:10.021Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004282,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1482gitlab # [ 214.567794] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:10.030Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"eba47e1043da56448e935e87","created_at":"2026-09-01T10:35:06.812Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7e7fa144dc8fba2f400bff790f3a710a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.991Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-eba47e1043da56448e935e87: start","job_status":"start","queue_duration_s":3.038842,"scheduling_latency_s":3.038888}1483gitlab # [ 216.187584] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:11.786Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"eba47e1043da56448e935e87","created_at":"2026-09-01T10:35:06.812Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7e7fa144dc8fba2f400bff790f3a710a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:06.991Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-eba47e1043da56448e935e87: done: 1.756432 sec","job_status":"done","queue_duration_s":3.038842,"scheduling_latency_s":3.038888,"redis_calls":9,"redis_duration_s":0.08504099999999999,"redis_read_bytes":7,"redis_write_bytes":987,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.063787,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.021254,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":129,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":129,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.298,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.298,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.349878,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.012807214,"exclusive_lock_hold_duration_s":1.597665904,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":1.756432,"completed_at":"2026-09-01T10:35:11.786Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.836545,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1484gitlab # [ 216.266992] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:11.800Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"411237298f370276d30c4fc3","created_at":"2026-09-01T10:35:07.004Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb9e667f1f913abdefd67e4ed8d7b068","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:07.013Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-411237298f370276d30c4fc3: start","job_status":"start","queue_duration_s":4.787122,"scheduling_latency_s":4.78717}1485gitlab # [ 216.289054] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:11.850Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"411237298f370276d30c4fc3","created_at":"2026-09-01T10:35:07.004Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb9e667f1f913abdefd67e4ed8d7b068","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:07.013Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-411237298f370276d30c4fc3: done: 0.050632 sec","job_status":"done","queue_duration_s":4.787122,"scheduling_latency_s":4.78717,"redis_calls":10,"redis_duration_s":0.006659,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002855,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003804,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021475,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.010251862,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.050632,"completed_at":"2026-09-01T10:35:11.850Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006343,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1486gitlab # [ 216.322926] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:11.853Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"e7c8721da053d9ed2b173260","created_at":"2026-09-01T10:35:07.036Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1db41bc6d6bf0503ce788f97f2af8cc0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-01T10:35:07.216Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-e7c8721da053d9ed2b173260: start","job_status":"start","queue_duration_s":4.637187,"scheduling_latency_s":4.637229}1487gitlab # [ 216.324045] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:11.870Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1488gitlab # [ 216.324988] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:11.880Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"e7c8721da053d9ed2b173260","created_at":"2026-09-01T10:35:07.036Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1db41bc6d6bf0503ce788f97f2af8cc0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-01T10:35:07.216Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-e7c8721da053d9ed2b173260: done: 0.026724 sec","job_status":"done","queue_duration_s":4.637187,"scheduling_latency_s":4.637229,"redis_calls":7,"redis_duration_s":0.003763,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00284,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000923,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014276,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026724,"completed_at":"2026-09-01T10:35:11.880Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1489gitlab # [ 216.444114] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:11.883Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"c94e7b87c1123d8d4432a357","created_at":"2026-09-01T10:35:07.302Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fc7db648778ded611a24803419af8439","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-01T10:35:07.310Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-c94e7b87c1123d8d4432a357: start","job_status":"start","queue_duration_s":4.572384,"scheduling_latency_s":4.572429}1490gitlab # [ 216.445197] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:11.898Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1491gitlab # [ 216.457336] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:11.908Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"c94e7b87c1123d8d4432a357","created_at":"2026-09-01T10:35:07.302Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fc7db648778ded611a24803419af8439","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-01T10:35:07.310Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-c94e7b87c1123d8d4432a357: done: 0.025085 sec","job_status":"done","queue_duration_s":4.572384,"scheduling_latency_s":4.572429,"redis_calls":7,"redis_duration_s":0.003748,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00266,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001088,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014332,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025085,"completed_at":"2026-09-01T10:35:11.908Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1492gitlab # [ 216.585682] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:11.910Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"b3a198627e9c893fcd2dc37e","created_at":"2026-09-01T10:35:07.354Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f1355af3c6552de6b1a8552d88e93937","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:07.386Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-b3a198627e9c893fcd2dc37e: start","job_status":"start","queue_duration_s":4.52442,"scheduling_latency_s":4.524451}1493gitlab # [ 216.587209] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:12.000Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"b3a198627e9c893fcd2dc37e","created_at":"2026-09-01T10:35:07.354Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f1355af3c6552de6b1a8552d88e93937","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:07.386Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-b3a198627e9c893fcd2dc37e: done: 0.090086 sec","job_status":"done","queue_duration_s":4.52442,"scheduling_latency_s":4.524451,"redis_calls":11,"redis_duration_s":0.02812,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.006779,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.018615,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.001018,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001708,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.005594007,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.090086,"completed_at":"2026-09-01T10:35:12.000Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000596,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1494gitlab # [ 216.652235] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:12.017Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"0cf48f15c68c377520a9f80b","created_at":"2026-09-01T10:35:07.437Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81c1da4b3853feac4d32349cecbc3420","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:07.453Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-0cf48f15c68c377520a9f80b: start","job_status":"start","queue_duration_s":4.563774,"scheduling_latency_s":4.563828}1495gitlab # [ 216.653426] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:12.069Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"0cf48f15c68c377520a9f80b","created_at":"2026-09-01T10:35:07.437Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81c1da4b3853feac4d32349cecbc3420","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:07.453Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-0cf48f15c68c377520a9f80b: done: 0.052163 sec","job_status":"done","queue_duration_s":4.563774,"scheduling_latency_s":4.563828,"redis_calls":7,"redis_duration_s":0.014751,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003536,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.011215,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.052163,"completed_at":"2026-09-01T10:35:12.069Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1496gitlab # [ 216.832372] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:12.071Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"3de34ccd96d5ac52ef3f96c6","created_at":"2026-09-01T10:35:07.505Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a21aeaedfd51db710cd681b5912f1f56","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:07.620Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-3de34ccd96d5ac52ef3f96c6: start","job_status":"start","queue_duration_s":4.450653,"scheduling_latency_s":4.450695}1497gitlab # [ 216.832791] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:12.086Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1498gitlab # [ 217.205984] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:12.103Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"3de34ccd96d5ac52ef3f96c6","created_at":"2026-09-01T10:35:07.505Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a21aeaedfd51db710cd681b5912f1f56","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:07.620Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-3de34ccd96d5ac52ef3f96c6: done: 0.031764 sec","job_status":"done","queue_duration_s":4.450653,"scheduling_latency_s":4.450695,"redis_calls":7,"redis_duration_s":0.003029,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002046,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000983,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031764,"completed_at":"2026-09-01T10:35:12.103Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1499gitlab # [ 217.221622] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:12.106Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"748a3753fc0cc21c86aec660","created_at":"2026-09-01T10:35:07.642Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81975c32ba6ef49ce3161c5d163f4d6f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:07.676Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-748a3753fc0cc21c86aec660: start","job_status":"start","queue_duration_s":4.429626,"scheduling_latency_s":4.429667}1500gitlab # [ 217.236131] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:12.144Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1501gitlab # [ 217.242145] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:12.162Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"748a3753fc0cc21c86aec660","created_at":"2026-09-01T10:35:07.642Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81975c32ba6ef49ce3161c5d163f4d6f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:07.676Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-748a3753fc0cc21c86aec660: done: 0.056766 sec","job_status":"done","queue_duration_s":4.429626,"scheduling_latency_s":4.429667,"redis_calls":7,"redis_duration_s":0.015556,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004565,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.010991,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010919,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.056766,"completed_at":"2026-09-01T10:35:12.162Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1502gitlab # [ 217.362400] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:12.166Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"d694dcb488ba9bb98464981b","created_at":"2026-09-01T10:35:07.714Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ac315dff2e99e81647527b2f1695a06","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:07.734Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-d694dcb488ba9bb98464981b: start","job_status":"start","queue_duration_s":4.432201,"scheduling_latency_s":4.432251}1503gitlab # [ 217.513296] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:12.302Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1504gitlab # [ 217.514347] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:12.375Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"d694dcb488ba9bb98464981b","created_at":"2026-09-01T10:35:07.714Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ac315dff2e99e81647527b2f1695a06","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:07.734Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-d694dcb488ba9bb98464981b: done: 0.209757 sec","job_status":"done","queue_duration_s":4.432201,"scheduling_latency_s":4.432251,"redis_calls":7,"redis_duration_s":0.085092,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.066941,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.018151,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020909,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.209757,"completed_at":"2026-09-01T10:35:12.375Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1505gitlab # [ 217.517525] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:12.421Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"eddb5bf222f7fd2844550bb0","created_at":"2026-09-01T10:35:07.786Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"43919429310cba5db41057ccdc2af2bd","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:07.795Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-eddb5bf222f7fd2844550bb0: start","job_status":"start","queue_duration_s":4.625846,"scheduling_latency_s":4.625898}1506gitlab # [ 217.667546] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:12.513Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"eddb5bf222f7fd2844550bb0","created_at":"2026-09-01T10:35:07.786Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"43919429310cba5db41057ccdc2af2bd","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:07.795Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-eddb5bf222f7fd2844550bb0: done: 0.091586 sec","job_status":"done","queue_duration_s":4.625846,"scheduling_latency_s":4.625898,"redis_calls":7,"redis_duration_s":0.0027860000000000003,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002056,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00073,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015725,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.091586,"completed_at":"2026-09-01T10:35:12.512Z","load_balancing_strategy":"primary","db_duration_s":0.029985,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1507gitlab # [ 217.789845] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:12.528Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"b165bce830f66aeddd1f6951","created_at":"2026-09-01T10:35:07.978Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"06bc24e63a57cafdc616dad9e0d293bb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:07.990Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-b165bce830f66aeddd1f6951: start","job_status":"start","queue_duration_s":4.537535,"scheduling_latency_s":4.537591}1508gitlab # [ 217.790340] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:12.656Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"b165bce830f66aeddd1f6951","created_at":"2026-09-01T10:35:07.978Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"06bc24e63a57cafdc616dad9e0d293bb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:07.990Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-b165bce830f66aeddd1f6951: done: 0.140935 sec","job_status":"done","queue_duration_s":4.537535,"scheduling_latency_s":4.537591,"redis_calls":9,"redis_duration_s":0.020238,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.011451,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.008787,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.03,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036268,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.004253334,"exclusive_lock_hold_duration_s":0.064433329,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.140935,"completed_at":"2026-09-01T10:35:12.656Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.039528,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1509gitlab # [ 217.853051] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:12.660Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"bdb3ebfc77a1b7e26786a008","created_at":"2026-09-01T10:35:08.025Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"764865a6b01c28635ce673823cd89d3f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:08.036Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-bdb3ebfc77a1b7e26786a008: start","job_status":"start","queue_duration_s":4.623251,"scheduling_latency_s":4.623294}1510gitlab # [ 217.854279] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:13.082Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"bdb3ebfc77a1b7e26786a008","created_at":"2026-09-01T10:35:08.025Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"764865a6b01c28635ce673823cd89d3f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:08.036Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-bdb3ebfc77a1b7e26786a008: done: 0.422599 sec","job_status":"done","queue_duration_s":4.623251,"scheduling_latency_s":4.623294,"redis_calls":14,"redis_duration_s":0.009659,"redis_read_bytes":15,"redis_write_bytes":4026,"redis_queues_calls":2,"redis_queues_duration_s":0.000417,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.006258,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002984,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.100368,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.422599,"completed_at":"2026-09-01T10:35:13.082Z","load_balancing_strategy":"primary","db_duration_s":0.008107,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1511gitlab # [ 217.994287] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:13.085Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"655716ddbe96077fc29a3555","created_at":"2026-09-01T10:35:08.055Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:08.065Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-655716ddbe96077fc29a3555: start","job_status":"start","queue_duration_s":5.019825,"scheduling_latency_s":5.01986}1512gitlab # [ 218.001384] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:13.444Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"655716ddbe96077fc29a3555","created_at":"2026-09-01T10:35:08.055Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:08.065Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-655716ddbe96077fc29a3555: done: 0.359216 sec","job_status":"done","queue_duration_s":5.019825,"scheduling_latency_s":5.01986,"redis_calls":60,"redis_duration_s":0.063099,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.004073,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.057553,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001473,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.158987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.359216,"completed_at":"2026-09-01T10:35:13.444Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1513gitlab # [ 218.065097] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:13.448Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"222f8ceee217ce6273513dd9","created_at":"2026-09-01T10:35:08.153Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a0ff221fccd1c1fb73f95f76e30e48d","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:08.162Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-222f8ceee217ce6273513dd9: start","job_status":"start","queue_duration_s":5.28536,"scheduling_latency_s":5.285568}1514gitlab # [ 218.066414] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:13.529Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"222f8ceee217ce6273513dd9","created_at":"2026-09-01T10:35:08.153Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a0ff221fccd1c1fb73f95f76e30e48d","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:08.162Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-222f8ceee217ce6273513dd9: done: 0.08181 sec","job_status":"done","queue_duration_s":5.28536,"scheduling_latency_s":5.285568,"redis_calls":7,"redis_duration_s":0.004238,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003433,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000805,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025286,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.08181,"completed_at":"2026-09-01T10:35:13.529Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015032,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1515gitlab # [ 218.169807] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:13.538Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"9a58fb9df4d9761d1e01bbc6","created_at":"2026-09-01T10:35:08.275Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4a2f2189b6e7276923bddab1b31385ad","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:08.283Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-9a58fb9df4d9761d1e01bbc6: start","job_status":"start","queue_duration_s":5.254335,"scheduling_latency_s":5.254385}1516gitlab # [ 218.172886] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:13.608Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"9a58fb9df4d9761d1e01bbc6","created_at":"2026-09-01T10:35:08.275Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4a2f2189b6e7276923bddab1b31385ad","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:08.283Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-9a58fb9df4d9761d1e01bbc6: done: 0.070922 sec","job_status":"done","queue_duration_s":5.254335,"scheduling_latency_s":5.254385,"redis_calls":9,"redis_duration_s":0.025319,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.013165,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.012154,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023408,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00718527,"exclusive_lock_hold_duration_s":0.009277436,"duration_s":0.070922,"completed_at":"2026-09-01T10:35:13.608Z","load_balancing_strategy":"primary","db_duration_s":0.001995,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1517gitlab # [ 218.263368] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:13.622Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"e04de6240badc2be22778796","created_at":"2026-09-01T10:35:08.333Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da4697052ad64a8bd89cf0afedba0ff2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:08.431Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-e04de6240badc2be22778796: start","job_status":"start","queue_duration_s":5.190813,"scheduling_latency_s":5.190856}1518gitlab # [ 218.360955] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:13.705Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"e04de6240badc2be22778796","created_at":"2026-09-01T10:35:08.333Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da4697052ad64a8bd89cf0afedba0ff2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-01T10:35:08.431Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-e04de6240badc2be22778796: done: 0.08273 sec","job_status":"done","queue_duration_s":5.190813,"scheduling_latency_s":5.190856,"redis_calls":9,"redis_duration_s":0.006246,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002695,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003551,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032274,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001329499,"exclusive_lock_hold_duration_s":0.043474519,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.08273,"completed_at":"2026-09-01T10:35:13.704Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.035363,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1519gitlab # [ 218.366312] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:13.711Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f367e9e9b748b5c6ff1542be","created_at":"2026-09-01T10:35:12.695Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-4c04da0ed6834da2","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"764865a6b01c28635ce673823cd89d3f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.073Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f367e9e9b748b5c6ff1542be: start","job_status":"start","queue_duration_s":0.638093,"scheduling_latency_s":0.638137}1520gitlab # [ 218.436324] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:13.735Z","message":"No existing merge request to be cleaned up."}1521gitlab # [ 218.438491] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:13.761Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f367e9e9b748b5c6ff1542be","created_at":"2026-09-01T10:35:12.695Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-4c04da0ed6834da2","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"764865a6b01c28635ce673823cd89d3f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.073Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f367e9e9b748b5c6ff1542be: done: 0.049759 sec","job_status":"done","queue_duration_s":0.638093,"scheduling_latency_s":0.638137,"redis_calls":10,"redis_duration_s":0.009961,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00329,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.006671,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023471,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.049759,"completed_at":"2026-09-01T10:35:13.761Z","load_balancing_strategy":"primary","db_duration_s":0.003253,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1522gitlab # [ 218.562191] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:13.774Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a008f88f3219ae39b11de566","created_at":"2026-09-01T10:35:12.695Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-4c04da0ed6834da2","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"764865a6b01c28635ce673823cd89d3f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.073Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a008f88f3219ae39b11de566: start","job_status":"start","queue_duration_s":0.700794,"scheduling_latency_s":0.700843}1523gitlab # [ 218.562916] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:13.831Z","message":"No existing merge request to be cleaned up."}1524gitlab # [ 218.575339] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:13.865Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a008f88f3219ae39b11de566","created_at":"2026-09-01T10:35:12.695Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-4c04da0ed6834da2","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"764865a6b01c28635ce673823cd89d3f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.073Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a008f88f3219ae39b11de566: done: 0.091139 sec","job_status":"done","queue_duration_s":0.700794,"scheduling_latency_s":0.700843,"redis_calls":10,"redis_duration_s":0.036283,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.012244,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.024039,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027779,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.091139,"completed_at":"2026-09-01T10:35:13.865Z","load_balancing_strategy":"primary","db_duration_s":0.005962,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1525gitlab # [ 218.585063] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:13.875Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"93cf54947e5a268bc0f4f445","created_at":"2026-09-01T10:35:12.695Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-4c04da0ed6834da2","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"764865a6b01c28635ce673823cd89d3f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.073Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-93cf54947e5a268bc0f4f445: start","job_status":"start","queue_duration_s":0.801464,"scheduling_latency_s":0.801508}1526gitlab # [ 218.633152] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:13.899Z","message":"No existing merge request to be cleaned up."}1527gitlab # [ 218.633540] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:13.925Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"93cf54947e5a268bc0f4f445","created_at":"2026-09-01T10:35:12.695Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-4c04da0ed6834da2","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"764865a6b01c28635ce673823cd89d3f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.073Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-93cf54947e5a268bc0f4f445: done: 0.050095 sec","job_status":"done","queue_duration_s":0.801464,"scheduling_latency_s":0.801508,"redis_calls":10,"redis_duration_s":0.011215,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003658,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.007557,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.050095,"completed_at":"2026-09-01T10:35:13.925Z","load_balancing_strategy":"primary","db_duration_s":0.007275,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1528gitlab # [ 218.728623] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:13.928Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7e1af6dfd3d195670b317aee","created_at":"2026-09-01T10:35:12.695Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-4c04da0ed6834da2","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"764865a6b01c28635ce673823cd89d3f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.073Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7e1af6dfd3d195670b317aee: start","job_status":"start","queue_duration_s":0.85459,"scheduling_latency_s":0.854628}1529gitlab # [ 218.740812] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:13.950Z","message":"No existing merge request to be cleaned up."}1530gitlab # [ 218.741901] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:13.968Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7e1af6dfd3d195670b317aee","created_at":"2026-09-01T10:35:12.695Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-4c04da0ed6834da2","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"764865a6b01c28635ce673823cd89d3f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.073Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7e1af6dfd3d195670b317aee: done: 0.040614 sec","job_status":"done","queue_duration_s":0.85459,"scheduling_latency_s":0.854628,"redis_calls":10,"redis_duration_s":0.00611,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002626,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003484,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040614,"completed_at":"2026-09-01T10:35:13.968Z","load_balancing_strategy":"primary","db_duration_s":0.002426,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1531gitlab # [ 218.816487] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:13.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81027f6b51d979b187a60e36","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81027f6b51d979b187a60e36: start","job_status":"start","queue_duration_s":0.629542,"scheduling_latency_s":0.629592}1532gitlab # [ 218.816843] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:14.079Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81027f6b51d979b187a60e36","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81027f6b51d979b187a60e36: done: 0.099624 sec","job_status":"done","queue_duration_s":0.629542,"scheduling_latency_s":0.629592,"redis_calls":13,"redis_duration_s":0.032865,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.008866,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.023999,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028865,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.099624,"completed_at":"2026-09-01T10:35:14.078Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1533gitlab # [ 218.942933] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:14.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61dd88c49cf04ffbb3a3b33f","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61dd88c49cf04ffbb3a3b33f: start","job_status":"start","queue_duration_s":0.733205,"scheduling_latency_s":0.733258}1534gitlab # [ 218.943368] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:14.136Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61dd88c49cf04ffbb3a3b33f","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61dd88c49cf04ffbb3a3b33f: done: 0.053388 sec","job_status":"done","queue_duration_s":0.733205,"scheduling_latency_s":0.733258,"redis_calls":13,"redis_duration_s":0.004821,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001643,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003178,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.053388,"completed_at":"2026-09-01T10:35:14.136Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1535gitlab # [ 219.047609] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:14.138Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa1cb38632a5181a6bb23307","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa1cb38632a5181a6bb23307: start","job_status":"start","queue_duration_s":0.789123,"scheduling_latency_s":0.789165}1536gitlab # [ 219.051184] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:14.277Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa1cb38632a5181a6bb23307","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa1cb38632a5181a6bb23307: done: 0.138923 sec","job_status":"done","queue_duration_s":0.789123,"scheduling_latency_s":0.789165,"redis_calls":13,"redis_duration_s":0.06435199999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.020831,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.043521,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02814,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.138923,"completed_at":"2026-09-01T10:35:14.277Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1537gitlab # [ 219.262722] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:14.289Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5292eaca655e001c8ce0a8c","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5292eaca655e001c8ce0a8c: start","job_status":"start","queue_duration_s":0.939627,"scheduling_latency_s":0.939677}1538gitlab # [ 219.263846] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:14.347Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5292eaca655e001c8ce0a8c","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5292eaca655e001c8ce0a8c: done: 0.058027 sec","job_status":"done","queue_duration_s":0.939627,"scheduling_latency_s":0.939677,"redis_calls":13,"redis_duration_s":0.012809,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.006166,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006643,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023289,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.058027,"completed_at":"2026-09-01T10:35:14.347Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1539gitlab # [ 219.499220] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:14.352Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23df2b16e05820ddff34dd0e","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23df2b16e05820ddff34dd0e: start","job_status":"start","queue_duration_s":1.002261,"scheduling_latency_s":1.002307}1540gitlab # [ 219.500974] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:14.465Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23df2b16e05820ddff34dd0e","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23df2b16e05820ddff34dd0e: done: 0.113305 sec","job_status":"done","queue_duration_s":1.002261,"scheduling_latency_s":1.002307,"redis_calls":13,"redis_duration_s":0.065181,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.049297,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.015884,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.113305,"completed_at":"2026-09-01T10:35:14.465Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1541gitlab # [ 219.611888] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:14.470Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"649e5fc3c5fa4fd981e7d5bb","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-649e5fc3c5fa4fd981e7d5bb: start","job_status":"start","queue_duration_s":1.120272,"scheduling_latency_s":1.120322}1542gitlab # [ 219.614584] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:14.548Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"649e5fc3c5fa4fd981e7d5bb","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-649e5fc3c5fa4fd981e7d5bb: done: 0.078609 sec","job_status":"done","queue_duration_s":1.120272,"scheduling_latency_s":1.120322,"redis_calls":13,"redis_duration_s":0.008253,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00348,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004773,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.078609,"completed_at":"2026-09-01T10:35:14.548Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1543gitlab # [ 219.882065] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:14.552Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ddd4542a18a6354f3d38d0b0","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ddd4542a18a6354f3d38d0b0: start","job_status":"start","queue_duration_s":1.202571,"scheduling_latency_s":1.202622}1544gitlab # [ 219.976643] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:14.585Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ddd4542a18a6354f3d38d0b0","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ddd4542a18a6354f3d38d0b0: done: 0.033307 sec","job_status":"done","queue_duration_s":1.202571,"scheduling_latency_s":1.202622,"redis_calls":13,"redis_duration_s":0.006751,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003233,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033307,"completed_at":"2026-09-01T10:35:14.585Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1545gitlab # [ 220.055907] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:15.655Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6bb9b7e9150850a042b53c19","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6bb9b7e9150850a042b53c19: start","job_status":"start","queue_duration_s":2.306092,"scheduling_latency_s":2.306146}1546gitlab # [ 220.217462] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:15.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6bb9b7e9150850a042b53c19","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6bb9b7e9150850a042b53c19: done: 0.160395 sec","job_status":"done","queue_duration_s":2.306092,"scheduling_latency_s":2.306146,"redis_calls":13,"redis_duration_s":0.046543,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00862,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.037923,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036768,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.160395,"completed_at":"2026-09-01T10:35:15.816Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1547gitlab # [ 220.295337] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:15.891Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc1c2c4ccdd2ccf4bcd6cfb7","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc1c2c4ccdd2ccf4bcd6cfb7: start","job_status":"start","queue_duration_s":2.541502,"scheduling_latency_s":2.541552}1548gitlab # [ 220.437417] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:16.034Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc1c2c4ccdd2ccf4bcd6cfb7","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc1c2c4ccdd2ccf4bcd6cfb7: done: 0.14347 sec","job_status":"done","queue_duration_s":2.541502,"scheduling_latency_s":2.541552,"redis_calls":13,"redis_duration_s":0.041679,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.012629,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.02905,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.14347,"completed_at":"2026-09-01T10:35:16.034Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1549gitlab # [ 220.578845] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:16.177Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"160db4b882229960df66aa52","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-160db4b882229960df66aa52: start","job_status":"start","queue_duration_s":2.827261,"scheduling_latency_s":2.827309}1550gitlab # [ 220.718788] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:16.318Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"160db4b882229960df66aa52","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-160db4b882229960df66aa52: done: 0.141469 sec","job_status":"done","queue_duration_s":2.827261,"scheduling_latency_s":2.827309,"redis_calls":13,"redis_duration_s":0.035851,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.011153,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.024698,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02509,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.141469,"completed_at":"2026-09-01T10:35:16.318Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1551gitlab # [ 221.133649] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:16.344Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ab60fd7b268ef2c3aaedc45","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ab60fd7b268ef2c3aaedc45: start","job_status":"start","queue_duration_s":2.995096,"scheduling_latency_s":2.995142}1552gitlab # [ 221.133957] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:16.729Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ab60fd7b268ef2c3aaedc45","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ab60fd7b268ef2c3aaedc45: done: 0.384993 sec","job_status":"done","queue_duration_s":2.995096,"scheduling_latency_s":2.995142,"redis_calls":13,"redis_duration_s":0.14132499999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.051939,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.089386,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026638,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.384993,"completed_at":"2026-09-01T10:35:16.729Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1553gitlab # [ 221.544454] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:16.741Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8dd1dbb2fd39bb7ac292485","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8dd1dbb2fd39bb7ac292485: start","job_status":"start","queue_duration_s":3.391182,"scheduling_latency_s":3.391226}1554gitlab # [ 221.544781] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:16.966Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8dd1dbb2fd39bb7ac292485","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8dd1dbb2fd39bb7ac292485: done: 0.225587 sec","job_status":"done","queue_duration_s":3.391182,"scheduling_latency_s":3.391226,"redis_calls":13,"redis_duration_s":0.0076,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004399,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003201,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02347,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.225587,"completed_at":"2026-09-01T10:35:16.966Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1555gitlab # [ 221.988524] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:16.968Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8179032c3b159cc82c89ef5","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8179032c3b159cc82c89ef5: start","job_status":"start","queue_duration_s":3.618683,"scheduling_latency_s":3.618714}1556gitlab # [ 221.990050] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:16.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8179032c3b159cc82c89ef5","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8179032c3b159cc82c89ef5: done: 0.026781 sec","job_status":"done","queue_duration_s":3.618683,"scheduling_latency_s":3.618714,"redis_calls":13,"redis_duration_s":0.003729,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001212,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002517,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015829,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026781,"completed_at":"2026-09-01T10:35:16.995Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1557gitlab # [ 222.155278] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:16.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"630085845925debb40085c19","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-630085845925debb40085c19: start","job_status":"start","queue_duration_s":3.647683,"scheduling_latency_s":3.647719}1558gitlab # [ 222.155605] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:17.319Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"630085845925debb40085c19","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-630085845925debb40085c19: done: 0.322382 sec","job_status":"done","queue_duration_s":3.647683,"scheduling_latency_s":3.647719,"redis_calls":13,"redis_duration_s":0.023123,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003706,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.019417,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.096522,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.322382,"completed_at":"2026-09-01T10:35:17.319Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1559gitlab # [ 222.427448] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:17.326Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c2781055d3e557a51b10a62","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c2781055d3e557a51b10a62: start","job_status":"start","queue_duration_s":3.977119,"scheduling_latency_s":3.977164}1560gitlab # [ 222.612327] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:18.207Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c2781055d3e557a51b10a62","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c2781055d3e557a51b10a62: done: 0.880315 sec","job_status":"done","queue_duration_s":3.977119,"scheduling_latency_s":3.977164,"redis_calls":13,"redis_duration_s":0.015475,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002745,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.01273,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036421,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.880315,"completed_at":"2026-09-01T10:35:18.207Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1561gitlab # [ 222.628339] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:18.223Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23019cd4382d662d5aec32d4","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23019cd4382d662d5aec32d4: start","job_status":"start","queue_duration_s":4.873645,"scheduling_latency_s":4.873691}1562gitlab # [ 222.765551] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:18.343Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23019cd4382d662d5aec32d4","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23019cd4382d662d5aec32d4: done: 0.11985 sec","job_status":"done","queue_duration_s":4.873645,"scheduling_latency_s":4.873691,"redis_calls":13,"redis_duration_s":0.066601,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.008641,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.05796,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026602,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.11985,"completed_at":"2026-09-01T10:35:18.343Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1563gitlab # [ 222.832952] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:18.347Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f66f2a5723992944e67338f","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f66f2a5723992944e67338f: start","job_status":"start","queue_duration_s":4.997604,"scheduling_latency_s":4.997657}1564gitlab # [ 222.837943] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:18.400Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f66f2a5723992944e67338f","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f66f2a5723992944e67338f: done: 0.052892 sec","job_status":"done","queue_duration_s":4.997604,"scheduling_latency_s":4.997657,"redis_calls":13,"redis_duration_s":0.01593,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.012836,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020514,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.052892,"completed_at":"2026-09-01T10:35:18.400Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1565gitlab # [ 222.915670] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:18.403Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"874e318eeb6a6bc55ea3f082","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-874e318eeb6a6bc55ea3f082: start","job_status":"start","queue_duration_s":5.053343,"scheduling_latency_s":5.053385}1566gitlab # [ 222.917423] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:18.444Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"874e318eeb6a6bc55ea3f082","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-874e318eeb6a6bc55ea3f082: done: 0.041321 sec","job_status":"done","queue_duration_s":5.053343,"scheduling_latency_s":5.053385,"redis_calls":13,"redis_duration_s":0.00792,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002104,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005816,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.041321,"completed_at":"2026-09-01T10:35:18.444Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1567gitlab # [ 223.042277] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:18.449Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"674bb816baece6cd0880053a","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-674bb816baece6cd0880053a: start","job_status":"start","queue_duration_s":5.099951,"scheduling_latency_s":5.100001}1568gitlab # [ 223.042597] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:18.505Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"674bb816baece6cd0880053a","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-674bb816baece6cd0880053a: done: 0.055323 sec","job_status":"done","queue_duration_s":5.099951,"scheduling_latency_s":5.100001,"redis_calls":13,"redis_duration_s":0.014630999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004873,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.009758,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.055323,"completed_at":"2026-09-01T10:35:18.504Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1569gitlab # [ 223.153472] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:18.524Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3813bcb2b88a674c5777a33d","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3813bcb2b88a674c5777a33d: start","job_status":"start","queue_duration_s":5.174999,"scheduling_latency_s":5.175099}1570gitlab # [ 223.256140] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:18.850Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3813bcb2b88a674c5777a33d","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3813bcb2b88a674c5777a33d: done: 0.338725 sec","job_status":"done","queue_duration_s":5.174999,"scheduling_latency_s":5.175099,"redis_calls":13,"redis_duration_s":0.063882,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.025738,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.038144,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.06405,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.338725,"completed_at":"2026-09-01T10:35:18.849Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1571gitlab # [ 223.272076] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:18.870Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a44666a409bf816d1e477525","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a44666a409bf816d1e477525: start","job_status":"start","queue_duration_s":5.520296,"scheduling_latency_s":5.520346}1572gitlab # [ 223.530093] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.129Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a44666a409bf816d1e477525","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a44666a409bf816d1e477525: done: 0.259491 sec","job_status":"done","queue_duration_s":5.520296,"scheduling_latency_s":5.520346,"redis_calls":13,"redis_duration_s":0.050303,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.014838,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.035465,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.259491,"completed_at":"2026-09-01T10:35:19.129Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1573gitlab # [ 223.694359] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.292Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0fc9b61cb7529a320d859a43","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0fc9b61cb7529a320d859a43: start","job_status":"start","queue_duration_s":5.942447,"scheduling_latency_s":5.942494}1574gitlab # [ 223.770898] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.368Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0fc9b61cb7529a320d859a43","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0fc9b61cb7529a320d859a43: done: 0.07664 sec","job_status":"done","queue_duration_s":5.942447,"scheduling_latency_s":5.942494,"redis_calls":13,"redis_duration_s":0.009947000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003972,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005975,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021286,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.07664,"completed_at":"2026-09-01T10:35:19.368Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1575gitlab # [ 223.772861] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.371Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5a9c04b6b70ef83928105004","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5a9c04b6b70ef83928105004: start","job_status":"start","queue_duration_s":6.021529,"scheduling_latency_s":6.021564}1576gitlab # [ 223.917545] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5a9c04b6b70ef83928105004","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5a9c04b6b70ef83928105004: done: 0.059444 sec","job_status":"done","queue_duration_s":6.021529,"scheduling_latency_s":6.021564,"redis_calls":13,"redis_duration_s":0.019975,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.014103,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022368,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.059444,"completed_at":"2026-09-01T10:35:19.430Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1577gitlab # [ 223.925072] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.439Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee7f697b7b098eb79dd2b8e8","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee7f697b7b098eb79dd2b8e8: start","job_status":"start","queue_duration_s":6.089323,"scheduling_latency_s":6.08937}1578gitlab # [ 224.008569] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.546Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee7f697b7b098eb79dd2b8e8","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee7f697b7b098eb79dd2b8e8: done: 0.107691 sec","job_status":"done","queue_duration_s":6.089323,"scheduling_latency_s":6.08937,"redis_calls":13,"redis_duration_s":0.050138,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.027871,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.022267,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.107691,"completed_at":"2026-09-01T10:35:19.546Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1579gitlab # [ 224.079567] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.550Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14f632a120bb8c2c7b4ea53d","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14f632a120bb8c2c7b4ea53d: start","job_status":"start","queue_duration_s":6.20082,"scheduling_latency_s":6.200898}1580gitlab # [ 224.080998] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.596Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14f632a120bb8c2c7b4ea53d","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14f632a120bb8c2c7b4ea53d: done: 0.045529 sec","job_status":"done","queue_duration_s":6.20082,"scheduling_latency_s":6.200898,"redis_calls":13,"redis_duration_s":0.008418,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002954,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005464,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023725,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.045529,"completed_at":"2026-09-01T10:35:19.595Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1581gitlab # [ 224.265287] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.599Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21efbc30543ca525a486bded","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21efbc30543ca525a486bded: start","job_status":"start","queue_duration_s":6.249623,"scheduling_latency_s":6.249672}1582gitlab # [ 224.266773] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.640Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21efbc30543ca525a486bded","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21efbc30543ca525a486bded: done: 0.04086 sec","job_status":"done","queue_duration_s":6.249623,"scheduling_latency_s":6.249672,"redis_calls":13,"redis_duration_s":0.007358,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002441,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0208,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.04086,"completed_at":"2026-09-01T10:35:19.640Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1583gitlab # [ 224.292381] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.644Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5ce91f7a5c40ed07f17f134","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5ce91f7a5c40ed07f17f134: start","job_status":"start","queue_duration_s":6.294438,"scheduling_latency_s":6.29449}1584gitlab # [ 224.390301] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.727Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5ce91f7a5c40ed07f17f134","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5ce91f7a5c40ed07f17f134: done: 0.082984 sec","job_status":"done","queue_duration_s":6.294438,"scheduling_latency_s":6.29449,"redis_calls":13,"redis_duration_s":0.013509,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004466,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.009043,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023195,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.082984,"completed_at":"2026-09-01T10:35:19.727Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1585gitlab # [ 224.393400] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.750Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2cb30683d7a6efbdf055d195","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2cb30683d7a6efbdf055d195: start","job_status":"start","queue_duration_s":6.40022,"scheduling_latency_s":6.400276}1586gitlab # [ 224.482054] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.786Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2cb30683d7a6efbdf055d195","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2cb30683d7a6efbdf055d195: done: 0.036218 sec","job_status":"done","queue_duration_s":6.40022,"scheduling_latency_s":6.400276,"redis_calls":13,"redis_duration_s":0.006956,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002157,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004799,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020496,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.036218,"completed_at":"2026-09-01T10:35:19.786Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1587gitlab # [ 224.489668] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"459e9c9b2d656211c49b332f","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-459e9c9b2d656211c49b332f: start","job_status":"start","queue_duration_s":6.439537,"scheduling_latency_s":6.439584}1588gitlab # [ 224.576764] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.824Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"459e9c9b2d656211c49b332f","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-459e9c9b2d656211c49b332f: done: 0.03531 sec","job_status":"done","queue_duration_s":6.439537,"scheduling_latency_s":6.439584,"redis_calls":13,"redis_duration_s":0.007098999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001737,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005362,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03531,"completed_at":"2026-09-01T10:35:19.824Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1589gitlab # [ 224.582421] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.827Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"94e380b4d3996fc51aa173c1","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-94e380b4d3996fc51aa173c1: start","job_status":"start","queue_duration_s":6.477902,"scheduling_latency_s":6.477945}1590gitlab # [ 224.666484] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.862Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"94e380b4d3996fc51aa173c1","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-94e380b4d3996fc51aa173c1: done: 0.035145 sec","job_status":"done","queue_duration_s":6.477902,"scheduling_latency_s":6.477945,"redis_calls":13,"redis_duration_s":0.0065520000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002232,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00432,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019376,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035145,"completed_at":"2026-09-01T10:35:19.862Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1591gitlab # [ 224.666875] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.869Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"181ec3cc70b2f5456d17361c","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-181ec3cc70b2f5456d17361c: start","job_status":"start","queue_duration_s":6.519759,"scheduling_latency_s":6.519811}1592gitlab # [ 224.819510] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.905Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"181ec3cc70b2f5456d17361c","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-181ec3cc70b2f5456d17361c: done: 0.036507 sec","job_status":"done","queue_duration_s":6.519759,"scheduling_latency_s":6.519811,"redis_calls":13,"redis_duration_s":0.009238,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00296,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006278,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017416,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.036507,"completed_at":"2026-09-01T10:35:19.905Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1593gitlab # [ 224.984166] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4c37794c65e9f3c3a4dc55b","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4c37794c65e9f3c3a4dc55b: start","job_status":"start","queue_duration_s":6.565266,"scheduling_latency_s":6.565324}1594gitlab # [ 224.985450] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.986Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4c37794c65e9f3c3a4dc55b","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4c37794c65e9f3c3a4dc55b: done: 0.071879 sec","job_status":"done","queue_duration_s":6.565266,"scheduling_latency_s":6.565324,"redis_calls":13,"redis_duration_s":0.008296,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003854,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004442,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.071879,"completed_at":"2026-09-01T10:35:19.986Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1595gitlab # [ 225.070308] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:19.989Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0fa46212d6146155d460529f","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0fa46212d6146155d460529f: start","job_status":"start","queue_duration_s":6.639556,"scheduling_latency_s":6.639608}1596gitlab # [ 225.071869] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:20.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0fa46212d6146155d460529f","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0fa46212d6146155d460529f: done: 0.044065 sec","job_status":"done","queue_duration_s":6.639556,"scheduling_latency_s":6.639608,"redis_calls":13,"redis_duration_s":0.008517,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003045,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005472,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.044065,"completed_at":"2026-09-01T10:35:20.033Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1597gitlab # [ 225.107204] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:20.037Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3cd485cc06be1d0fb58b745e","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3cd485cc06be1d0fb58b745e: start","job_status":"start","queue_duration_s":6.687456,"scheduling_latency_s":6.687505}1598gitlab # [ 225.125166] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:20.068Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3cd485cc06be1d0fb58b745e","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3cd485cc06be1d0fb58b745e: done: 0.031214 sec","job_status":"done","queue_duration_s":6.687456,"scheduling_latency_s":6.687505,"redis_calls":13,"redis_duration_s":0.005175,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002072,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003103,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01945,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031214,"completed_at":"2026-09-01T10:35:20.068Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1599gitlab # [ 225.237072] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:20.070Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55e0b18fcd662d4045ab2648","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55e0b18fcd662d4045ab2648: start","job_status":"start","queue_duration_s":6.721163,"scheduling_latency_s":6.721206}1600gitlab # [ 225.238483] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:20.103Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55e0b18fcd662d4045ab2648","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55e0b18fcd662d4045ab2648: done: 0.032567 sec","job_status":"done","queue_duration_s":6.721163,"scheduling_latency_s":6.721206,"redis_calls":13,"redis_duration_s":0.007366,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004693,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002673,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032567,"completed_at":"2026-09-01T10:35:20.103Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1601gitlab # [ 225.355158] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:20.105Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce0a54f503bc44b82e3a4d06","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce0a54f503bc44b82e3a4d06: start","job_status":"start","queue_duration_s":6.75612,"scheduling_latency_s":6.75616}1602gitlab # [ 225.356531] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:20.133Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce0a54f503bc44b82e3a4d06","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce0a54f503bc44b82e3a4d06: done: 0.026621 sec","job_status":"done","queue_duration_s":6.75612,"scheduling_latency_s":6.75616,"redis_calls":13,"redis_duration_s":0.004474,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001174,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026621,"completed_at":"2026-09-01T10:35:20.132Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1603gitlab # [ 225.453061] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:20.135Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f94b468bbad8e7174e753ca","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f94b468bbad8e7174e753ca: start","job_status":"start","queue_duration_s":6.785826,"scheduling_latency_s":6.785858}1604gitlab # [ 225.493319] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:20.163Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f94b468bbad8e7174e753ca","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f94b468bbad8e7174e753ca: done: 0.027792 sec","job_status":"done","queue_duration_s":6.785826,"scheduling_latency_s":6.785858,"redis_calls":13,"redis_duration_s":0.004802,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001212,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00359,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017492,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027792,"completed_at":"2026-09-01T10:35:20.163Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1605gitlab # [ 225.829889] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:20.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a21eb8f16778d70499fec833","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a21eb8f16778d70499fec833: start","job_status":"start","queue_duration_s":6.81599,"scheduling_latency_s":6.816066}1606gitlab # [ 225.844337] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:20.204Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a21eb8f16778d70499fec833","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a21eb8f16778d70499fec833: done: 0.038672 sec","job_status":"done","queue_duration_s":6.81599,"scheduling_latency_s":6.816066,"redis_calls":13,"redis_duration_s":0.007705999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004696,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00301,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038672,"completed_at":"2026-09-01T10:35:20.204Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1607gitlab # [ 225.930464] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:20.208Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be58a2f6cc53cc0fbf6a1adf","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be58a2f6cc53cc0fbf6a1adf: start","job_status":"start","queue_duration_s":6.858536,"scheduling_latency_s":6.858579}1608gitlab # [ 225.932183] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:20.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be58a2f6cc53cc0fbf6a1adf","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be58a2f6cc53cc0fbf6a1adf: done: 0.061558 sec","job_status":"done","queue_duration_s":6.858536,"scheduling_latency_s":6.858579,"redis_calls":13,"redis_duration_s":0.017409,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.014659,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00275,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017434,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.061558,"completed_at":"2026-09-01T10:35:20.269Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1609gitlab # [ 226.367517] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:21.967Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a21f0d62e2153c84d53d0917","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a21f0d62e2153c84d53d0917: start","job_status":"start","queue_duration_s":8.617918,"scheduling_latency_s":8.617961}1610gitlab # [ 226.586357] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:22.185Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a21f0d62e2153c84d53d0917","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a21f0d62e2153c84d53d0917: done: 0.217505 sec","job_status":"done","queue_duration_s":8.617918,"scheduling_latency_s":8.617961,"redis_calls":13,"redis_duration_s":0.0075699999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002959,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004611,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019917,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.217505,"completed_at":"2026-09-01T10:35:22.185Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1611gitlab # [ 226.643650] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:22.193Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45af0046a126b3ea9d359bc1","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45af0046a126b3ea9d359bc1: start","job_status":"start","queue_duration_s":8.8433,"scheduling_latency_s":8.843352}1612gitlab # [ 226.662100] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:22.229Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45af0046a126b3ea9d359bc1","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45af0046a126b3ea9d359bc1: done: 0.036952 sec","job_status":"done","queue_duration_s":8.8433,"scheduling_latency_s":8.843352,"redis_calls":13,"redis_duration_s":0.006689,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002169,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00452,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019634,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.036952,"completed_at":"2026-09-01T10:35:22.229Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1613gitlab # [ 226.725229] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:22.233Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57acd5e69eb221e7dcd7723d","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57acd5e69eb221e7dcd7723d: start","job_status":"start","queue_duration_s":8.883283,"scheduling_latency_s":8.883328}1614gitlab # [ 226.749257] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:22.266Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57acd5e69eb221e7dcd7723d","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57acd5e69eb221e7dcd7723d: done: 0.033058 sec","job_status":"done","queue_duration_s":8.883283,"scheduling_latency_s":8.883328,"redis_calls":13,"redis_duration_s":0.007389,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002239,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00515,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018821,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033058,"completed_at":"2026-09-01T10:35:22.265Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1615gitlab # [ 226.783963] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:22.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e012a0f9187914663588d950","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e012a0f9187914663588d950: start","job_status":"start","queue_duration_s":8.919545,"scheduling_latency_s":8.91959}1616gitlab # [ 226.785371] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:22.301Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e012a0f9187914663588d950","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e012a0f9187914663588d950: done: 0.0321 sec","job_status":"done","queue_duration_s":8.919545,"scheduling_latency_s":8.91959,"redis_calls":13,"redis_duration_s":0.006817,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00243,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004387,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018233,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0321,"completed_at":"2026-09-01T10:35:22.301Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1617gitlab # [ 226.873948] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:22.304Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11a2814b4b146d85be1edbc0","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11a2814b4b146d85be1edbc0: start","job_status":"start","queue_duration_s":8.954563,"scheduling_latency_s":8.954602}1618gitlab # [ 226.876318] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:22.377Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11a2814b4b146d85be1edbc0","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11a2814b4b146d85be1edbc0: done: 0.073208 sec","job_status":"done","queue_duration_s":8.954563,"scheduling_latency_s":8.954602,"redis_calls":13,"redis_duration_s":0.004934,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001705,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003229,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.059642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.073208,"completed_at":"2026-09-01T10:35:22.377Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1619gitlab # [ 227.043312] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:22.380Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbfce7f8ae799dfbd8c14633","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbfce7f8ae799dfbd8c14633: start","job_status":"start","queue_duration_s":9.030641,"scheduling_latency_s":9.03068}1620gitlab # [ 227.045534] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:22.413Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbfce7f8ae799dfbd8c14633","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbfce7f8ae799dfbd8c14633: done: 0.032986 sec","job_status":"done","queue_duration_s":9.030641,"scheduling_latency_s":9.03068,"redis_calls":13,"redis_duration_s":0.005632,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001443,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004189,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032986,"completed_at":"2026-09-01T10:35:22.413Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1621gitlab # [ 227.132727] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:22.416Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eef89dca0c006923d12cf091","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eef89dca0c006923d12cf091: start","job_status":"start","queue_duration_s":9.066405,"scheduling_latency_s":9.066439}1622gitlab # [ 227.134115] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:22.468Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eef89dca0c006923d12cf091","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eef89dca0c006923d12cf091: done: 0.052817 sec","job_status":"done","queue_duration_s":9.066405,"scheduling_latency_s":9.066439,"redis_calls":13,"redis_duration_s":0.020622,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.006371,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.014251,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.052817,"completed_at":"2026-09-01T10:35:22.468Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1623gitlab # [ 227.170545] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:22.480Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31baac96efcd3e21340f446c","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31baac96efcd3e21340f446c: start","job_status":"start","queue_duration_s":9.130337,"scheduling_latency_s":9.130389}1624gitlab # [ 227.231504] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:22.624Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31baac96efcd3e21340f446c","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31baac96efcd3e21340f446c: done: 0.144375 sec","job_status":"done","queue_duration_s":9.130337,"scheduling_latency_s":9.130389,"redis_calls":13,"redis_duration_s":0.051805,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.043502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008303,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027915,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.144375,"completed_at":"2026-09-01T10:35:22.624Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1625gitlab # [ 227.240103] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:22.628Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e039671309d52843cd22acb","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e039671309d52843cd22acb: start","job_status":"start","queue_duration_s":9.278285,"scheduling_latency_s":9.278326}1626gitlab # [ 227.259743] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:22.675Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e039671309d52843cd22acb","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e039671309d52843cd22acb: done: 0.047485 sec","job_status":"done","queue_duration_s":9.278285,"scheduling_latency_s":9.278326,"redis_calls":13,"redis_duration_s":0.009232,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001712,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00752,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020503,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.047485,"completed_at":"2026-09-01T10:35:22.675Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1627gitlab # [ 227.342292] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:22.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bb4ee002870e1ccd578ec50","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bb4ee002870e1ccd578ec50: start","job_status":"start","queue_duration_s":9.328223,"scheduling_latency_s":9.328258}1628gitlab # [ 227.419086] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:22.708Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bb4ee002870e1ccd578ec50","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bb4ee002870e1ccd578ec50: done: 0.030226 sec","job_status":"done","queue_duration_s":9.328223,"scheduling_latency_s":9.328258,"redis_calls":13,"redis_duration_s":0.006272999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001473,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0048,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017882,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030226,"completed_at":"2026-09-01T10:35:22.708Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1629gitlab # [ 227.435530] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:22.710Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7218b63426645485aec647ea","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7218b63426645485aec647ea: start","job_status":"start","queue_duration_s":9.361144,"scheduling_latency_s":9.361177}1630gitlab # [ 227.497516] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:35:22.760Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7218b63426645485aec647ea","created_at":"2026-09-01T10:35:13.104Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-1d54a856d44b48de","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"de86414df8a4f641f2d75666094181c3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:35:13.349Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7218b63426645485aec647ea: done: 0.049668 sec","job_status":"done","queue_duration_s":9.361144,"scheduling_latency_s":9.361177,"redis_calls":13,"redis_duration_s":0.019131000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001392,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.017739,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.049668,"completed_at":"2026-09-01T10:35:22.760Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1631gitlab # [ 245.455756] setup-gitlab-start[2583]: W, [2026-09-01T10:35:41.055273 #2583] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1632gitlab # [ 268.870872] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:04.470Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","jid":"36091e003862600707eced4d","created_at":"2026-09-01T10:36:04.452Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a45550fda7383c03816a6a38b545e469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:805014656d48c4962f83ad00758ac8cfab4de244cfbd1d771454e66bd9018911","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.468Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker JID-36091e003862600707eced4d: start","job_status":"start","queue_duration_s":0.002349,"scheduling_latency_s":0.002431}1633gitlab # [ 269.003139] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:04.603Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","jid":"36091e003862600707eced4d","created_at":"2026-09-01T10:36:04.452Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a45550fda7383c03816a6a38b545e469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:805014656d48c4962f83ad00758ac8cfab4de244cfbd1d771454e66bd9018911","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.468Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker JID-36091e003862600707eced4d: done: 0.132958 sec","job_status":"done","queue_duration_s":0.002349,"scheduling_latency_s":0.002431,"redis_calls":27,"redis_duration_s":0.048639,"redis_read_bytes":791,"redis_write_bytes":3651,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.03756,"redis_feature_flag_read_bytes":780,"redis_feature_flag_write_bytes":1750,"redis_queues_calls":2,"redis_queues_duration_s":0.001707,"redis_queues_read_bytes":2,"redis_queues_write_bytes":817,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.006822,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":734,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00255,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":350,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045669,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000674387,"duration_s":0.132958,"completed_at":"2026-09-01T10:36:04.603Z","load_balancing_strategy":"primary","db_duration_s":0.005501,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1634gitlab # [ 269.009754] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:04.610Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"440e8ef116c3a1c923a2eba2","created_at":"2026-09-01T10:36:04.481Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d33d3282fcca59fe780b2d0c245bed7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.487Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-440e8ef116c3a1c923a2eba2: start","job_status":"start","queue_duration_s":0.122933,"scheduling_latency_s":0.12297}1635gitlab # [ 269.065874] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:04.665Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"440e8ef116c3a1c923a2eba2","created_at":"2026-09-01T10:36:04.481Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d33d3282fcca59fe780b2d0c245bed7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.487Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-440e8ef116c3a1c923a2eba2: done: 0.055795 sec","job_status":"done","queue_duration_s":0.122933,"scheduling_latency_s":0.12297,"redis_calls":10,"redis_duration_s":0.029265,"redis_read_bytes":389,"redis_write_bytes":1078,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.016567,"redis_feature_flag_read_bytes":384,"redis_feature_flag_write_bytes":247,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.011526,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001172,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.055795,"completed_at":"2026-09-01T10:36:04.665Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1636gitlab # [ 269.128420] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:04.672Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"1c99029c5a22a1ab37144cc8","created_at":"2026-09-01T10:36:04.505Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b1a9fa1ce6e450459499284683869f60","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.514Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-1c99029c5a22a1ab37144cc8: start","job_status":"start","queue_duration_s":0.157563,"scheduling_latency_s":0.157608}1637gitlab # [ 269.136898] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:04.737Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"1c99029c5a22a1ab37144cc8","created_at":"2026-09-01T10:36:04.505Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b1a9fa1ce6e450459499284683869f60","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.514Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-1c99029c5a22a1ab37144cc8: done: 0.064864 sec","job_status":"done","queue_duration_s":0.157563,"scheduling_latency_s":0.157608,"redis_calls":11,"redis_duration_s":0.015326,"redis_read_bytes":263,"redis_write_bytes":1276,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001213,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":149,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.009876,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004237,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018402,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001100977,"exclusive_lock_hold_duration_s":0.01788244,"duration_s":0.064864,"completed_at":"2026-09-01T10:36:04.737Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014579,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1638gitlab # [ 269.200225] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:04.744Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"94a95e24a8c751f201fd88ac","created_at":"2026-09-01T10:36:04.540Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c6fc7bdc1598e8426354e9eadfa11344","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.547Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-94a95e24a8c751f201fd88ac: start","job_status":"start","queue_duration_s":0.196393,"scheduling_latency_s":0.19643}1639gitlab # [ 269.201307] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:04.779Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1640gitlab # [ 269.202080] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:04.794Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"94a95e24a8c751f201fd88ac","created_at":"2026-09-01T10:36:04.540Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c6fc7bdc1598e8426354e9eadfa11344","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.547Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-94a95e24a8c751f201fd88ac: done: 0.050213 sec","job_status":"done","queue_duration_s":0.196393,"scheduling_latency_s":0.19643,"redis_calls":8,"redis_duration_s":0.014492000000000001,"redis_read_bytes":133,"redis_write_bytes":867,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000494,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":101,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005708,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00829,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013246,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.050213,"completed_at":"2026-09-01T10:36:04.794Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1641gitlab # [ 269.275214] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:04.799Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Projects::RefreshBuildArtifactsSizeStatisticsWorker","args":[],"jid":"23c360aeeb364d0814476bc5","created_at":"2026-09-01T10:36:04.554Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-45f97700ae0043c0","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","correlation_id":"a45550fda7383c03816a6a38b545e469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.572Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::RefreshBuildArtifactsSizeStatisticsWorker JID-23c360aeeb364d0814476bc5: start","job_status":"start","queue_duration_s":0.226791,"scheduling_latency_s":0.226836}1642gitlab # [ 269.303972] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:04.903Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Projects::RefreshBuildArtifactsSizeStatisticsWorker","args":[],"jid":"23c360aeeb364d0814476bc5","created_at":"2026-09-01T10:36:04.554Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-45f97700ae0043c0","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","correlation_id":"a45550fda7383c03816a6a38b545e469","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.572Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::RefreshBuildArtifactsSizeStatisticsWorker JID-23c360aeeb364d0814476bc5: done: 0.104042 sec","job_status":"done","queue_duration_s":0.226791,"scheduling_latency_s":0.226836,"redis_calls":13,"redis_duration_s":0.01381,"redis_read_bytes":12,"redis_write_bytes":1665,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002588,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006129,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":519,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.005093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":801,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.015,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.015,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034246,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000924699,"duration_s":0.104042,"completed_at":"2026-09-01T10:36:04.903Z","load_balancing_strategy":"primary","db_duration_s":0.020645,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1643gitlab # [ 269.367727] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:04.911Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Runners::StaleMachinesCleanupCronWorker","jid":"207afa324842756edd5ad994","created_at":"2026-09-01T10:36:04.559Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"637e0523d147adcb9869be734d493981","meta.root_caller_id":"Cronjob","meta.feature_category":"fleet_visibility","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b74a831dd4b14c3fc2b303407a0e209ea633ae1f03f57327e51b9e7be5871f1f","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.577Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Runners::StaleMachinesCleanupCronWorker JID-207afa324842756edd5ad994: start","job_status":"start","queue_duration_s":0.33401,"scheduling_latency_s":0.334062}1644gitlab # [ 269.403415] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:05.003Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Runners::StaleMachinesCleanupCronWorker","jid":"207afa324842756edd5ad994","created_at":"2026-09-01T10:36:04.559Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"637e0523d147adcb9869be734d493981","meta.root_caller_id":"Cronjob","meta.feature_category":"fleet_visibility","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b74a831dd4b14c3fc2b303407a0e209ea633ae1f03f57327e51b9e7be5871f1f","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.577Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Runners::StaleMachinesCleanupCronWorker JID-207afa324842756edd5ad994: done: 0.091903 sec","job_status":"done","queue_duration_s":0.33401,"scheduling_latency_s":0.334062,"redis_calls":10,"redis_duration_s":0.012625,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001646,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008843,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002136,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.027,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025828,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0012848,"extra.ci_runners_stale_machines_cleanup_cron_worker.status":"success","extra.ci_runners_stale_machines_cleanup_cron_worker.total_deleted":0,"extra.ci_runners_stale_machines_cleanup_cron_worker.batch_counts":[0],"duration_s":0.091903,"completed_at":"2026-09-01T10:36:05.003Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.040638,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1645gitlab # [ 269.415368] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:05.015Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"63f3e64993358bc59d8e75db","created_at":"2026-09-01T10:36:04.600Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"a90a7167059e5b281adda6ae87091438","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.609Z","job_size_bytes":3,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-63f3e64993358bc59d8e75db: start","job_status":"start","queue_duration_s":0.406015,"scheduling_latency_s":0.406086}1646gitlab # [ 269.521767] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:05.121Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"63f3e64993358bc59d8e75db","created_at":"2026-09-01T10:36:04.600Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"a90a7167059e5b281adda6ae87091438","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.609Z","job_size_bytes":3,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-63f3e64993358bc59d8e75db: done: 0.106331 sec","job_status":"done","queue_duration_s":0.406015,"scheduling_latency_s":0.406086,"redis_calls":10,"redis_duration_s":0.007430000000000001,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00119,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003435,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002805,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000511797,"duration_s":0.106331,"completed_at":"2026-09-01T10:36:05.121Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005005,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1647gitlab # [ 269.525564] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:05.125Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"c14ad6960e28892b8b04c5e1","created_at":"2026-09-01T10:36:04.622Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"68c5cecbac56988c2069a3cec997dc93","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-01T10:36:04.641Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-c14ad6960e28892b8b04c5e1: start","job_status":"start","queue_duration_s":0.484513,"scheduling_latency_s":0.484553}1648gitlab # [ 269.569142] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:05.168Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"c14ad6960e28892b8b04c5e1","created_at":"2026-09-01T10:36:04.622Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"68c5cecbac56988c2069a3cec997dc93","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-01T10:36:04.641Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-c14ad6960e28892b8b04c5e1: done: 0.042948 sec","job_status":"done","queue_duration_s":0.484513,"scheduling_latency_s":0.484553,"redis_calls":8,"redis_duration_s":0.003833,"redis_read_bytes":133,"redis_write_bytes":836,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000534,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":95,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002537,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000762,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022825,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.042948,"completed_at":"2026-09-01T10:36:05.168Z","load_balancing_strategy":"primary","db_duration_s":0.002219,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1649gitlab # [ 269.634315] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:05.171Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ae2bcb81b8f281bb0ce01217","created_at":"2026-09-01T10:36:04.659Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4feec23db8fbc5ed572c31cace03ba76","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.669Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ae2bcb81b8f281bb0ce01217: start","job_status":"start","queue_duration_s":0.502704,"scheduling_latency_s":0.502749}1650gitlab # [ 269.635395] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:05.207Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ae2bcb81b8f281bb0ce01217","created_at":"2026-09-01T10:36:04.659Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4feec23db8fbc5ed572c31cace03ba76","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.669Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ae2bcb81b8f281bb0ce01217: done: 0.035771 sec","job_status":"done","queue_duration_s":0.502704,"scheduling_latency_s":0.502749,"redis_calls":10,"redis_duration_s":0.004533000000000001,"redis_read_bytes":135,"redis_write_bytes":1184,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000348,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":97,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00234,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001845,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001406045,"exclusive_lock_hold_duration_s":0.005581994,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.035771,"completed_at":"2026-09-01T10:36:05.207Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001771,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1651gitlab # [ 269.707448] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:05.210Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"47dc09101e3dabd5a450380d","created_at":"2026-09-01T10:36:04.681Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"555b916599d85d2af5331c7bfda73270","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.695Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-47dc09101e3dabd5a450380d: start","job_status":"start","queue_duration_s":0.514534,"scheduling_latency_s":0.514578}1652gitlab # [ 269.708781] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:05.246Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"47dc09101e3dabd5a450380d","created_at":"2026-09-01T10:36:04.681Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"555b916599d85d2af5331c7bfda73270","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.695Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-47dc09101e3dabd5a450380d: done: 0.036356 sec","job_status":"done","queue_duration_s":0.514534,"scheduling_latency_s":0.514578,"redis_calls":12,"redis_duration_s":0.004726,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000696,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002465,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001565,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021248,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00060706,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.036356,"completed_at":"2026-09-01T10:36:05.246Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000939,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1653gitlab # [ 269.772271] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:05.249Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"35355bddee782bcf2c9a365e","created_at":"2026-09-01T10:36:04.720Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0b95d0c0d43d9eb9ee12be12a790ef04","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.726Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-35355bddee782bcf2c9a365e: start","job_status":"start","queue_duration_s":0.522427,"scheduling_latency_s":0.522468}1654gitlab # [ 270.349806] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:05.949Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"35355bddee782bcf2c9a365e","created_at":"2026-09-01T10:36:04.720Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0b95d0c0d43d9eb9ee12be12a790ef04","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.726Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-35355bddee782bcf2c9a365e: done: 0.700145 sec","job_status":"done","queue_duration_s":0.522427,"scheduling_latency_s":0.522468,"redis_calls":563,"redis_duration_s":0.21260099999999998,"redis_read_bytes":688,"redis_write_bytes":48843,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000336,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":83,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001876,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.210389,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.530719,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.700145,"completed_at":"2026-09-01T10:36:05.949Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1655gitlab # [ 270.412509] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.012Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"2c7b8873627b4fc78b480b0b","created_at":"2026-09-01T10:36:04.734Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d78547f36f48a106df447113d0b3cf30","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.740Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-2c7b8873627b4fc78b480b0b: start","job_status":"start","queue_duration_s":1.271863,"scheduling_latency_s":1.271901}1656gitlab # [ 270.456131] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.056Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"2c7b8873627b4fc78b480b0b","created_at":"2026-09-01T10:36:04.734Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d78547f36f48a106df447113d0b3cf30","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.740Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-2c7b8873627b4fc78b480b0b: done: 0.043452 sec","job_status":"done","queue_duration_s":1.271863,"scheduling_latency_s":1.271901,"redis_calls":8,"redis_duration_s":0.003447,"redis_read_bytes":133,"redis_write_bytes":764,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000362,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":83,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002092,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000993,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043452,"completed_at":"2026-09-01T10:36:06.056Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000891,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1657gitlab # [ 270.520166] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.119Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"1000e9122bad91b4237c8329","created_at":"2026-09-01T10:36:04.753Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d585b1b2fdf44588b818cbb216426c8f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.769Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-1000e9122bad91b4237c8329: start","job_status":"start","queue_duration_s":1.34945,"scheduling_latency_s":1.349494}1658gitlab # [ 270.828367] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.428Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"1000e9122bad91b4237c8329","created_at":"2026-09-01T10:36:04.753Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d585b1b2fdf44588b818cbb216426c8f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.769Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-1000e9122bad91b4237c8329: done: 0.309044 sec","job_status":"done","queue_duration_s":1.34945,"scheduling_latency_s":1.349494,"redis_calls":11,"redis_duration_s":0.0049559999999999995,"redis_read_bytes":263,"redis_write_bytes":1140,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000783,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":153,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001773,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.0024,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":129,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":129,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.052,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":129,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.052,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.233457,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000869943,"exclusive_lock_hold_duration_s":0.282118816,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.309044,"completed_at":"2026-09-01T10:36:06.428Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.079471,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1659gitlab # [ 270.893842] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.493Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"327619faf751ac9b65a5d35f","created_at":"2026-09-01T10:36:04.783Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b4ac5616a7637fc64ee443e51f9e789b","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.792Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-327619faf751ac9b65a5d35f: start","job_status":"start","queue_duration_s":1.701315,"scheduling_latency_s":1.701358}1660gitlab # [ 270.928485] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.527Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"327619faf751ac9b65a5d35f","created_at":"2026-09-01T10:36:04.783Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b4ac5616a7637fc64ee443e51f9e789b","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.792Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-327619faf751ac9b65a5d35f: done: 0.033856 sec","job_status":"done","queue_duration_s":1.701315,"scheduling_latency_s":1.701358,"redis_calls":11,"redis_duration_s":0.004289,"redis_read_bytes":136,"redis_write_bytes":1339,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.00035,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":100,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002353,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001586,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021158,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.006139328,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.033856,"completed_at":"2026-09-01T10:36:06.527Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003032,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1661gitlab # [ 270.930639] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.531Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"1c08ee6a2766f7c9ffd3c1b3","created_at":"2026-09-01T10:36:04.800Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"75f31e84c883bd03f07b68395a1c22fb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-01T10:36:04.804Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-1c08ee6a2766f7c9ffd3c1b3: start","job_status":"start","queue_duration_s":1.726196,"scheduling_latency_s":1.726236}1662gitlab # [ 271.007238] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.544Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1663gitlab # [ 271.008307] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.554Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"1c08ee6a2766f7c9ffd3c1b3","created_at":"2026-09-01T10:36:04.800Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"75f31e84c883bd03f07b68395a1c22fb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-01T10:36:04.804Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-1c08ee6a2766f7c9ffd3c1b3: done: 0.023296 sec","job_status":"done","queue_duration_s":1.726196,"scheduling_latency_s":1.726236,"redis_calls":8,"redis_duration_s":0.002624,"redis_read_bytes":133,"redis_write_bytes":908,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000269,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":107,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00164,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000715,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01487,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023296,"completed_at":"2026-09-01T10:36:06.554Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1664gitlab # [ 271.011383] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.557Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"06cafefe1e03b73349965e44","created_at":"2026-09-01T10:36:04.822Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c2de4cb3131ee09d272a12d50ddd27e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-01T10:36:04.832Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-06cafefe1e03b73349965e44: start","job_status":"start","queue_duration_s":1.724873,"scheduling_latency_s":1.724909}1665gitlab # [ 271.074398] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.571Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1666gitlab # [ 271.075801] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.579Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"06cafefe1e03b73349965e44","created_at":"2026-09-01T10:36:04.822Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c2de4cb3131ee09d272a12d50ddd27e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-01T10:36:04.832Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-06cafefe1e03b73349965e44: done: 0.022393 sec","job_status":"done","queue_duration_s":1.724873,"scheduling_latency_s":1.724909,"redis_calls":8,"redis_duration_s":0.002678,"redis_read_bytes":133,"redis_write_bytes":914,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000313,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":108,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001731,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000634,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014565,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022393,"completed_at":"2026-09-01T10:36:06.579Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1667gitlab # [ 271.148047] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.582Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"2509b7ccc6d3cd9b99d44cca","created_at":"2026-09-01T10:36:04.860Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0842bb707f1c55208d77eb79b15594a8","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.866Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-2509b7ccc6d3cd9b99d44cca: start","job_status":"start","queue_duration_s":1.716262,"scheduling_latency_s":1.716302}1668gitlab # [ 271.149340] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.612Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"2509b7ccc6d3cd9b99d44cca","created_at":"2026-09-01T10:36:04.860Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0842bb707f1c55208d77eb79b15594a8","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.866Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-2509b7ccc6d3cd9b99d44cca: done: 0.030068 sec","job_status":"done","queue_duration_s":1.716262,"scheduling_latency_s":1.716302,"redis_calls":8,"redis_duration_s":0.004152,"redis_read_bytes":133,"redis_write_bytes":888,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000253,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":102,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002866,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001033,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030068,"completed_at":"2026-09-01T10:36:06.612Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001576,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1669gitlab # [ 271.213196] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.616Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"2230149b11e01d058d0ddd81","created_at":"2026-09-01T10:36:04.879Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3cb359d6c6c71a2bc3d1e808fd4bcc6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.890Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-2230149b11e01d058d0ddd81: start","job_status":"start","queue_duration_s":1.726285,"scheduling_latency_s":1.726327}1670gitlab # [ 271.214856] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.638Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"2230149b11e01d058d0ddd81","created_at":"2026-09-01T10:36:04.879Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3cb359d6c6c71a2bc3d1e808fd4bcc6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.890Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-2230149b11e01d058d0ddd81: done: 0.021898 sec","job_status":"done","queue_duration_s":1.726285,"scheduling_latency_s":1.726327,"redis_calls":8,"redis_duration_s":0.002392,"redis_read_bytes":133,"redis_write_bytes":879,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000228,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":103,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001532,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000632,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01309,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021898,"completed_at":"2026-09-01T10:36:06.638Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1671gitlab # [ 271.297616] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.640Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"7827fa598e7312632894e949","created_at":"2026-09-01T10:36:04.905Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"45069ab3dd7148d8b6c4266d30f41d1e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.910Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-7827fa598e7312632894e949: start","job_status":"start","queue_duration_s":1.729753,"scheduling_latency_s":1.729788}1672gitlab # [ 271.303209] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.653Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1673gitlab # [ 271.304223] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.662Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"7827fa598e7312632894e949","created_at":"2026-09-01T10:36:04.905Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"45069ab3dd7148d8b6c4266d30f41d1e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.910Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-7827fa598e7312632894e949: done: 0.022042 sec","job_status":"done","queue_duration_s":1.729753,"scheduling_latency_s":1.729788,"redis_calls":8,"redis_duration_s":0.003144,"redis_read_bytes":133,"redis_write_bytes":873,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000389,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":102,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002018,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000737,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013686,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022042,"completed_at":"2026-09-01T10:36:06.662Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1674gitlab # [ 271.373647] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.664Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"cf0c8482d14567ae69737097","created_at":"2026-09-01T10:36:04.923Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9227e030a626e7086e3845ac90fcdb1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.930Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-cf0c8482d14567ae69737097: start","job_status":"start","queue_duration_s":1.733927,"scheduling_latency_s":1.733959}1675gitlab # [ 271.376497] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.678Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1676gitlab # [ 271.378520] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.686Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"cf0c8482d14567ae69737097","created_at":"2026-09-01T10:36:04.923Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9227e030a626e7086e3845ac90fcdb1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.930Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-cf0c8482d14567ae69737097: done: 0.021682 sec","job_status":"done","queue_duration_s":1.733927,"scheduling_latency_s":1.733959,"redis_calls":8,"redis_duration_s":0.002728,"redis_read_bytes":133,"redis_write_bytes":937,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000415,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":111,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001629,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000684,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013219,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021682,"completed_at":"2026-09-01T10:36:06.686Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1677gitlab # [ 271.444912] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.688Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"21aeca9d1a0348e86cc39baa","created_at":"2026-09-01T10:36:04.953Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"45a4e8e658bc56546b46fab77671bd12","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.964Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-21aeca9d1a0348e86cc39baa: start","job_status":"start","queue_duration_s":1.723539,"scheduling_latency_s":1.723573}1678gitlab # [ 271.537520] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.701Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1679gitlab # [ 271.539622] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.707Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"21aeca9d1a0348e86cc39baa","created_at":"2026-09-01T10:36:04.953Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"45a4e8e658bc56546b46fab77671bd12","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.964Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-21aeca9d1a0348e86cc39baa: done: 0.019144 sec","job_status":"done","queue_duration_s":1.723539,"scheduling_latency_s":1.723573,"redis_calls":8,"redis_duration_s":0.002309,"redis_read_bytes":133,"redis_write_bytes":931,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000222,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":110,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00133,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000757,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011835,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019144,"completed_at":"2026-09-01T10:36:06.707Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1680gitlab # [ 271.542926] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.709Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"1f81418e48446bec2e7cda23","created_at":"2026-09-01T10:36:04.973Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d17f09b0631b557a7b3b22fd1389947","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.976Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-1f81418e48446bec2e7cda23: start","job_status":"start","queue_duration_s":1.73267,"scheduling_latency_s":1.732708}1681gitlab # [ 271.563414] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.791Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"1f81418e48446bec2e7cda23","created_at":"2026-09-01T10:36:04.973Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d17f09b0631b557a7b3b22fd1389947","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.976Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-1f81418e48446bec2e7cda23: done: 0.082436 sec","job_status":"done","queue_duration_s":1.73267,"scheduling_latency_s":1.732708,"redis_calls":8,"redis_duration_s":0.002243,"redis_read_bytes":133,"redis_write_bytes":936,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000189,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":110,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001482,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000572,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.062,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.062,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.082436,"completed_at":"2026-09-01T10:36:06.791Z","load_balancing_strategy":"primary","db_duration_s":0.062068,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1682gitlab # [ 271.637876] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.796Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"e6a14ac04c013bf96f9cf2b4","created_at":"2026-09-01T10:36:04.990Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"36c7feffd423bdf79ec90fd5e0618aa7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.995Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-e6a14ac04c013bf96f9cf2b4: start","job_status":"start","queue_duration_s":1.800646,"scheduling_latency_s":1.800685}1683gitlab # [ 271.645223] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.856Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"e6a14ac04c013bf96f9cf2b4","created_at":"2026-09-01T10:36:04.990Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"36c7feffd423bdf79ec90fd5e0618aa7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:04.995Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-e6a14ac04c013bf96f9cf2b4: done: 0.060418 sec","job_status":"done","queue_duration_s":1.800646,"scheduling_latency_s":1.800685,"redis_calls":11,"redis_duration_s":0.0053809999999999995,"redis_read_bytes":263,"redis_write_bytes":1334,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000638,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":172,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002613,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00213,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035854,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000876089,"exclusive_lock_hold_duration_s":0.02827622,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.060418,"completed_at":"2026-09-01T10:36:06.856Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01225,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1684gitlab # [ 271.713774] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.860Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"a647901c44a8afdb3bbfa173","created_at":"2026-09-01T10:36:05.011Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52a70e4b4e891063fdcb672eb384f9d4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.017Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-a647901c44a8afdb3bbfa173: start","job_status":"start","queue_duration_s":1.842909,"scheduling_latency_s":1.842958}1685gitlab # [ 271.781275] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.945Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"a647901c44a8afdb3bbfa173","created_at":"2026-09-01T10:36:05.011Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52a70e4b4e891063fdcb672eb384f9d4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.017Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-a647901c44a8afdb3bbfa173: done: 0.084543 sec","job_status":"done","queue_duration_s":1.842909,"scheduling_latency_s":1.842958,"redis_calls":16,"redis_duration_s":0.007274,"redis_read_bytes":271,"redis_write_bytes":4225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001221,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":199,"redis_queues_calls":2,"redis_queues_duration_s":0.000378,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.004227,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001448,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.056523,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.084543,"completed_at":"2026-09-01T10:36:06.945Z","load_balancing_strategy":"primary","db_duration_s":0.013346,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1686gitlab # [ 271.786532] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:06.947Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"63903a7fa7ce7ae5abe480dd","created_at":"2026-09-01T10:36:05.022Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.027Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-63903a7fa7ce7ae5abe480dd: start","job_status":"start","queue_duration_s":1.919432,"scheduling_latency_s":1.919464}1687gitlab # [ 271.807470] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:07.308Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"63903a7fa7ce7ae5abe480dd","created_at":"2026-09-01T10:36:05.022Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.027Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-63903a7fa7ce7ae5abe480dd: done: 0.360526 sec","job_status":"done","queue_duration_s":1.919432,"scheduling_latency_s":1.919464,"redis_calls":65,"redis_duration_s":0.12581299999999998,"redis_read_bytes":748,"redis_write_bytes":41502,"redis_feature_flag_calls":5,"redis_feature_flag_duration_s":0.001252,"redis_feature_flag_read_bytes":640,"redis_feature_flag_write_bytes":393,"redis_queues_calls":2,"redis_queues_duration_s":0.001569,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.120761,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002231,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.135525,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.360526,"completed_at":"2026-09-01T10:36:07.307Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1688gitlab # [ 271.878106] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:07.316Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"7b39a66e45309c5f5e3ea121","created_at":"2026-09-01T10:36:05.048Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb7f88458f4714df69cffec42c653ae7","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.053Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-7b39a66e45309c5f5e3ea121: start","job_status":"start","queue_duration_s":2.263336,"scheduling_latency_s":2.263376}1689gitlab # [ 271.892219] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:07.448Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"7b39a66e45309c5f5e3ea121","created_at":"2026-09-01T10:36:05.048Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb7f88458f4714df69cffec42c653ae7","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.053Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-7b39a66e45309c5f5e3ea121: done: 0.131906 sec","job_status":"done","queue_duration_s":2.263336,"scheduling_latency_s":2.263376,"redis_calls":11,"redis_duration_s":0.017023,"redis_read_bytes":517,"redis_write_bytes":1011,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.004666,"redis_feature_flag_read_bytes":512,"redis_feature_flag_write_bytes":300,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008931,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003426,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.131906,"completed_at":"2026-09-01T10:36:07.448Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.079643,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1690gitlab # [ 271.970878] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:07.452Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"df650075a70473ad4ee33cf2","created_at":"2026-09-01T10:36:05.057Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4047054966f64499f69eb8994f1275b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.061Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-df650075a70473ad4ee33cf2: start","job_status":"start","queue_duration_s":2.39066,"scheduling_latency_s":2.390695}1691gitlab # [ 271.972049] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:07.529Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"df650075a70473ad4ee33cf2","created_at":"2026-09-01T10:36:05.057Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4047054966f64499f69eb8994f1275b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.061Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-df650075a70473ad4ee33cf2: done: 0.077078 sec","job_status":"done","queue_duration_s":2.39066,"scheduling_latency_s":2.390695,"redis_calls":10,"redis_duration_s":0.022654,"redis_read_bytes":135,"redis_write_bytes":1167,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.001972,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":95,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.012314,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.008368,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.003200127,"exclusive_lock_hold_duration_s":0.031819966,"duration_s":0.077078,"completed_at":"2026-09-01T10:36:07.529Z","load_balancing_strategy":"primary","db_duration_s":0.028026,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1692gitlab # [ 272.037348] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:07.536Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"0ab477c89940e3af17dc5411","created_at":"2026-09-01T10:36:05.068Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"990bb74a3ed79d99fb30f65429dd856b","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.074Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-0ab477c89940e3af17dc5411: start","job_status":"start","queue_duration_s":2.461289,"scheduling_latency_s":2.461338}1693gitlab # [ 272.105200] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:07.572Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"0ab477c89940e3af17dc5411","created_at":"2026-09-01T10:36:05.068Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"990bb74a3ed79d99fb30f65429dd856b","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.074Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-0ab477c89940e3af17dc5411: done: 0.036697 sec","job_status":"done","queue_duration_s":2.461289,"scheduling_latency_s":2.461338,"redis_calls":11,"redis_duration_s":0.00418,"redis_read_bytes":138,"redis_write_bytes":877,"redis_cache_calls":3,"redis_cache_duration_s":0.000731,"redis_cache_read_bytes":5,"redis_cache_write_bytes":97,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.00024,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":84,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001505,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001704,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013404,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036697,"completed_at":"2026-09-01T10:36:07.572Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1694gitlab # [ 272.106995] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:07.598Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"575817a255ade05d8a815b8f","created_at":"2026-09-01T10:36:05.082Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fd850a24c5a4bac2835876687b268462","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.092Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-575817a255ade05d8a815b8f: start","job_status":"start","queue_duration_s":2.506069,"scheduling_latency_s":2.506117}1695gitlab # [ 272.322982] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:07.922Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"575817a255ade05d8a815b8f","created_at":"2026-09-01T10:36:05.082Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-dd3df680cf4a4ddb","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fd850a24c5a4bac2835876687b268462","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:05.092Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-575817a255ade05d8a815b8f: done: 0.324368 sec","job_status":"done","queue_duration_s":2.506069,"scheduling_latency_s":2.506117,"redis_calls":10,"redis_duration_s":0.222029,"redis_read_bytes":135,"redis_write_bytes":1275,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.091442,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":106,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.119297,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.01129,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030739,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00244193,"exclusive_lock_hold_duration_s":0.030040131,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.324368,"completed_at":"2026-09-01T10:36:07.922Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.032402,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1696gitlab # [ 272.401236] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.000Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"36e67e18aa85272f302dd791","created_at":"2026-09-01T10:36:06.882Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-923f6e858aec4bed","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"52a70e4b4e891063fdcb672eb384f9d4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:06.936Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-36e67e18aa85272f302dd791: start","job_status":"start","queue_duration_s":1.063566,"scheduling_latency_s":1.063716}1697gitlab # [ 272.445194] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.045Z","message":"No existing merge request to be cleaned up."}1698gitlab # [ 272.470116] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.070Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"36e67e18aa85272f302dd791","created_at":"2026-09-01T10:36:06.882Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-923f6e858aec4bed","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"52a70e4b4e891063fdcb672eb384f9d4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:06.936Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-36e67e18aa85272f302dd791: done: 0.069759 sec","job_status":"done","queue_duration_s":1.063566,"scheduling_latency_s":1.063716,"redis_calls":11,"redis_duration_s":0.021593,"redis_read_bytes":136,"redis_write_bytes":1104,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.002965,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":82,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.007207,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.011421,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024261,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.069759,"completed_at":"2026-09-01T10:36:08.070Z","load_balancing_strategy":"primary","db_duration_s":0.012436,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1699gitlab # [ 272.475480] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.075Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a72ecdbdf9d3b0407539362c","created_at":"2026-09-01T10:36:06.882Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-923f6e858aec4bed","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"52a70e4b4e891063fdcb672eb384f9d4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:06.936Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a72ecdbdf9d3b0407539362c: start","job_status":"start","queue_duration_s":1.138771,"scheduling_latency_s":1.138815}1700gitlab # [ 272.591672] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.192Z","message":"No existing merge request to be cleaned up."}1701gitlab # [ 272.639261] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.239Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a72ecdbdf9d3b0407539362c","created_at":"2026-09-01T10:36:06.882Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-923f6e858aec4bed","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"52a70e4b4e891063fdcb672eb384f9d4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:06.936Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a72ecdbdf9d3b0407539362c: done: 0.163586 sec","job_status":"done","queue_duration_s":1.138771,"scheduling_latency_s":1.138815,"redis_calls":10,"redis_duration_s":0.022611,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.014,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.008611,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.163586,"completed_at":"2026-09-01T10:36:08.239Z","load_balancing_strategy":"primary","db_duration_s":0.013824,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1702gitlab # [ 272.700710] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.244Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"524eb1b47ff101442c181105","created_at":"2026-09-01T10:36:06.882Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-923f6e858aec4bed","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"52a70e4b4e891063fdcb672eb384f9d4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:06.936Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-524eb1b47ff101442c181105: start","job_status":"start","queue_duration_s":1.307133,"scheduling_latency_s":1.307179}1703gitlab # [ 272.721434] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.270Z","message":"No existing merge request to be cleaned up."}1704gitlab # [ 272.724352] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.295Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"524eb1b47ff101442c181105","created_at":"2026-09-01T10:36:06.882Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-923f6e858aec4bed","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"52a70e4b4e891063fdcb672eb384f9d4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:06.936Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-524eb1b47ff101442c181105: done: 0.051296 sec","job_status":"done","queue_duration_s":1.307133,"scheduling_latency_s":1.307179,"redis_calls":10,"redis_duration_s":0.005156,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003254,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001902,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019238,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.051296,"completed_at":"2026-09-01T10:36:08.295Z","load_balancing_strategy":"primary","db_duration_s":0.012542,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1705gitlab # [ 272.791609] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.298Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c722596a98882254f2daf881","created_at":"2026-09-01T10:36:06.882Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-923f6e858aec4bed","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"52a70e4b4e891063fdcb672eb384f9d4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:06.936Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c722596a98882254f2daf881: start","job_status":"start","queue_duration_s":1.361378,"scheduling_latency_s":1.36142}1706gitlab # [ 272.810360] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.349Z","message":"No existing merge request to be cleaned up."}1707gitlab # [ 272.814938] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.370Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c722596a98882254f2daf881","created_at":"2026-09-01T10:36:06.882Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-923f6e858aec4bed","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"52a70e4b4e891063fdcb672eb384f9d4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:06.936Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c722596a98882254f2daf881: done: 0.072489 sec","job_status":"done","queue_duration_s":1.361378,"scheduling_latency_s":1.36142,"redis_calls":10,"redis_duration_s":0.020166,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.01069,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.009476,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022413,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.072489,"completed_at":"2026-09-01T10:36:08.370Z","load_balancing_strategy":"primary","db_duration_s":0.007484,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1708gitlab # [ 272.851160] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5cc99004c03c805d1ff44e76","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5cc99004c03c805d1ff44e76: start","job_status":"start","queue_duration_s":1.142848,"scheduling_latency_s":1.142895}1709gitlab # [ 272.948600] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.548Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5cc99004c03c805d1ff44e76","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5cc99004c03c805d1ff44e76: done: 0.118403 sec","job_status":"done","queue_duration_s":1.142848,"scheduling_latency_s":1.142895,"redis_calls":14,"redis_duration_s":0.018345,"redis_read_bytes":138,"redis_write_bytes":1242,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.0016,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":85,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.01221,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019491,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.118403,"completed_at":"2026-09-01T10:36:08.548Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1710gitlab # [ 272.958674] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.558Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b955a5954541d574ffefd860","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b955a5954541d574ffefd860: start","job_status":"start","queue_duration_s":1.271149,"scheduling_latency_s":1.271193}1711gitlab # [ 273.042390] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.642Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b955a5954541d574ffefd860","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b955a5954541d574ffefd860: done: 0.084023 sec","job_status":"done","queue_duration_s":1.271149,"scheduling_latency_s":1.271193,"redis_calls":13,"redis_duration_s":0.012591999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007032,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0176,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.084023,"completed_at":"2026-09-01T10:36:08.642Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1712gitlab # [ 273.062223] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.662Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97082f1553d49228fba03f04","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97082f1553d49228fba03f04: start","job_status":"start","queue_duration_s":1.374889,"scheduling_latency_s":1.374936}1713gitlab # [ 273.160435] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.760Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97082f1553d49228fba03f04","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97082f1553d49228fba03f04: done: 0.098022 sec","job_status":"done","queue_duration_s":1.374889,"scheduling_latency_s":1.374936,"redis_calls":13,"redis_duration_s":0.023462,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.010216,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.013246,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017188,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.098022,"completed_at":"2026-09-01T10:36:08.760Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1714gitlab # [ 273.252414] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.852Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d6ce2a3e7749fe0f8db1be1","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d6ce2a3e7749fe0f8db1be1: start","job_status":"start","queue_duration_s":1.56517,"scheduling_latency_s":1.565215}1715gitlab # [ 273.333863] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d6ce2a3e7749fe0f8db1be1","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d6ce2a3e7749fe0f8db1be1: done: 0.081232 sec","job_status":"done","queue_duration_s":1.56517,"scheduling_latency_s":1.565215,"redis_calls":13,"redis_duration_s":0.026978000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.010711,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.016267,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021178,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.081232,"completed_at":"2026-09-01T10:36:08.933Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1716gitlab # [ 273.335981] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.936Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f86ee909e4b6858c90ede179","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f86ee909e4b6858c90ede179: start","job_status":"start","queue_duration_s":1.648809,"scheduling_latency_s":1.648846}1717gitlab # [ 273.364198] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.962Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f86ee909e4b6858c90ede179","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f86ee909e4b6858c90ede179: done: 0.026604 sec","job_status":"done","queue_duration_s":1.648809,"scheduling_latency_s":1.648846,"redis_calls":13,"redis_duration_s":0.004562999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00306,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01661,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026604,"completed_at":"2026-09-01T10:36:08.962Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1718gitlab # [ 273.445724] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.965Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe34624dbb129abb8d7f80a1","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe34624dbb129abb8d7f80a1: start","job_status":"start","queue_duration_s":1.678312,"scheduling_latency_s":1.678388}1719gitlab # [ 273.448138] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe34624dbb129abb8d7f80a1","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe34624dbb129abb8d7f80a1: done: 0.028892 sec","job_status":"done","queue_duration_s":1.678312,"scheduling_latency_s":1.678388,"redis_calls":13,"redis_duration_s":0.006125,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002319,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003806,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028892,"completed_at":"2026-09-01T10:36:08.994Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1720gitlab # [ 273.516279] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:08.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34eca1054048835c618f2f30","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34eca1054048835c618f2f30: start","job_status":"start","queue_duration_s":1.709612,"scheduling_latency_s":1.70965}1721gitlab # [ 273.517505] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.026Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34eca1054048835c618f2f30","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34eca1054048835c618f2f30: done: 0.029139 sec","job_status":"done","queue_duration_s":1.709612,"scheduling_latency_s":1.70965,"redis_calls":13,"redis_duration_s":0.005662,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001648,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004014,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017541,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029139,"completed_at":"2026-09-01T10:36:09.026Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1722gitlab # [ 273.591253] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.028Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a6246752b46391f91601c47","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a6246752b46391f91601c47: start","job_status":"start","queue_duration_s":1.741162,"scheduling_latency_s":1.741201}1723gitlab # [ 273.593053] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.057Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a6246752b46391f91601c47","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a6246752b46391f91601c47: done: 0.02875 sec","job_status":"done","queue_duration_s":1.741162,"scheduling_latency_s":1.741201,"redis_calls":13,"redis_duration_s":0.0052239999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001631,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003593,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017809,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02875,"completed_at":"2026-09-01T10:36:09.057Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1724gitlab # [ 273.705413] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.060Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d28781a3c3b9c6c8af074a3a","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d28781a3c3b9c6c8af074a3a: start","job_status":"start","queue_duration_s":1.772762,"scheduling_latency_s":1.772803}1725gitlab # [ 273.706744] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.090Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d28781a3c3b9c6c8af074a3a","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d28781a3c3b9c6c8af074a3a: done: 0.030045 sec","job_status":"done","queue_duration_s":1.772762,"scheduling_latency_s":1.772803,"redis_calls":13,"redis_duration_s":0.006385999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001971,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004415,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018559,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030045,"completed_at":"2026-09-01T10:36:09.090Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1726gitlab # [ 273.734238] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.093Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"becfb3cfd8b4d98f95cc7d82","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-becfb3cfd8b4d98f95cc7d82: start","job_status":"start","queue_duration_s":1.805691,"scheduling_latency_s":1.805731}1727gitlab # [ 273.753461] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"becfb3cfd8b4d98f95cc7d82","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-becfb3cfd8b4d98f95cc7d82: done: 0.027179 sec","job_status":"done","queue_duration_s":1.805691,"scheduling_latency_s":1.805731,"redis_calls":13,"redis_duration_s":0.004953,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001618,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003335,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01683,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027179,"completed_at":"2026-09-01T10:36:09.120Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1728gitlab # [ 273.825791] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.122Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf18e83955e0b1c572bbaec8","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf18e83955e0b1c572bbaec8: start","job_status":"start","queue_duration_s":1.835017,"scheduling_latency_s":1.835061}1729gitlab # [ 273.839859] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf18e83955e0b1c572bbaec8","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf18e83955e0b1c572bbaec8: done: 0.033873 sec","job_status":"done","queue_duration_s":1.835017,"scheduling_latency_s":1.835061,"redis_calls":13,"redis_duration_s":0.006717,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002478,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004239,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019075,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033873,"completed_at":"2026-09-01T10:36:09.156Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1730gitlab # [ 273.915118] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.159Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a01aa4f26c25329974ddadd9","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a01aa4f26c25329974ddadd9: start","job_status":"start","queue_duration_s":1.872036,"scheduling_latency_s":1.872085}1731gitlab # [ 273.918134] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.190Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a01aa4f26c25329974ddadd9","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a01aa4f26c25329974ddadd9: done: 0.03131 sec","job_status":"done","queue_duration_s":1.872036,"scheduling_latency_s":1.872085,"redis_calls":13,"redis_duration_s":0.00586,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001652,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004208,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01867,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03131,"completed_at":"2026-09-01T10:36:09.190Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1732gitlab # [ 273.986562] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.194Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0270052bffb81a49910c3ea","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0270052bffb81a49910c3ea: start","job_status":"start","queue_duration_s":1.907423,"scheduling_latency_s":1.907468}1733gitlab # [ 273.993654] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.224Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0270052bffb81a49910c3ea","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0270052bffb81a49910c3ea: done: 0.029226 sec","job_status":"done","queue_duration_s":1.907423,"scheduling_latency_s":1.907468,"redis_calls":13,"redis_duration_s":0.004155,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001405,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00275,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017596,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029226,"completed_at":"2026-09-01T10:36:09.223Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1734gitlab # [ 274.062768] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.226Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64e0dd96c806fbc6b1c71c38","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64e0dd96c806fbc6b1c71c38: start","job_status":"start","queue_duration_s":1.939012,"scheduling_latency_s":1.939063}1735gitlab # [ 274.068265] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.254Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64e0dd96c806fbc6b1c71c38","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64e0dd96c806fbc6b1c71c38: done: 0.027876 sec","job_status":"done","queue_duration_s":1.939012,"scheduling_latency_s":1.939063,"redis_calls":13,"redis_duration_s":0.004174,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001298,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002876,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016709,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027876,"completed_at":"2026-09-01T10:36:09.254Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1736gitlab # [ 274.171163] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.256Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d74433be47dd2b6a8086523","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d74433be47dd2b6a8086523: start","job_status":"start","queue_duration_s":1.969202,"scheduling_latency_s":1.969239}1737gitlab # [ 274.172860] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.282Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d74433be47dd2b6a8086523","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d74433be47dd2b6a8086523: done: 0.025556 sec","job_status":"done","queue_duration_s":1.969202,"scheduling_latency_s":1.969239,"redis_calls":13,"redis_duration_s":0.003917,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001278,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002639,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01606,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025556,"completed_at":"2026-09-01T10:36:09.282Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1738gitlab # [ 274.242139] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.284Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76403e92f90a006eb4684d8e","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76403e92f90a006eb4684d8e: start","job_status":"start","queue_duration_s":1.99708,"scheduling_latency_s":1.997112}1739gitlab # [ 274.246291] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.310Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76403e92f90a006eb4684d8e","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76403e92f90a006eb4684d8e: done: 0.026229 sec","job_status":"done","queue_duration_s":1.99708,"scheduling_latency_s":1.997112,"redis_calls":13,"redis_duration_s":0.004208,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001312,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002896,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026229,"completed_at":"2026-09-01T10:36:09.310Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1740gitlab # [ 274.318357] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.313Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c51a8921593ba1c45d1cbf03","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c51a8921593ba1c45d1cbf03: start","job_status":"start","queue_duration_s":2.026401,"scheduling_latency_s":2.02646}1741gitlab # [ 274.319611] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.347Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c51a8921593ba1c45d1cbf03","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c51a8921593ba1c45d1cbf03: done: 0.033967 sec","job_status":"done","queue_duration_s":2.026401,"scheduling_latency_s":2.02646,"redis_calls":13,"redis_duration_s":0.005019,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003477,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017909,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033967,"completed_at":"2026-09-01T10:36:09.347Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1742gitlab # [ 274.361816] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.350Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c32d92d859a091acfc81c056","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c32d92d859a091acfc81c056: start","job_status":"start","queue_duration_s":2.063254,"scheduling_latency_s":2.063287}1743gitlab # [ 274.435389] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.377Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c32d92d859a091acfc81c056","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c32d92d859a091acfc81c056: done: 0.026649 sec","job_status":"done","queue_duration_s":2.063254,"scheduling_latency_s":2.063287,"redis_calls":13,"redis_duration_s":0.004018,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001184,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002834,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026649,"completed_at":"2026-09-01T10:36:09.377Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1744gitlab # [ 274.439356] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.379Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8a8f4f37cbf6cf375e8d6f0","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8a8f4f37cbf6cf375e8d6f0: start","job_status":"start","queue_duration_s":2.092221,"scheduling_latency_s":2.09226}1745gitlab # [ 274.459947] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.406Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8a8f4f37cbf6cf375e8d6f0","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8a8f4f37cbf6cf375e8d6f0: done: 0.026976 sec","job_status":"done","queue_duration_s":2.092221,"scheduling_latency_s":2.09226,"redis_calls":13,"redis_duration_s":0.004553,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001376,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003177,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026976,"completed_at":"2026-09-01T10:36:09.406Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1746gitlab # [ 274.539893] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.408Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7759312291587cd25fc9ef44","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7759312291587cd25fc9ef44: start","job_status":"start","queue_duration_s":2.121493,"scheduling_latency_s":2.121532}1747gitlab # [ 274.541556] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.437Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7759312291587cd25fc9ef44","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7759312291587cd25fc9ef44: done: 0.02836 sec","job_status":"done","queue_duration_s":2.121493,"scheduling_latency_s":2.121532,"redis_calls":13,"redis_duration_s":0.004459,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002896,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018562,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02836,"completed_at":"2026-09-01T10:36:09.437Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1748gitlab # [ 274.631224] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.439Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c52f058e2ba10494e15480a0","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c52f058e2ba10494e15480a0: start","job_status":"start","queue_duration_s":2.152069,"scheduling_latency_s":2.152105}1749gitlab # [ 274.633556] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.470Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c52f058e2ba10494e15480a0","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c52f058e2ba10494e15480a0: done: 0.030739 sec","job_status":"done","queue_duration_s":2.152069,"scheduling_latency_s":2.152105,"redis_calls":13,"redis_duration_s":0.004909,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001751,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003158,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030739,"completed_at":"2026-09-01T10:36:09.470Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1750gitlab # [ 274.711287] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.473Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53ea823d8ca6cef638c8df64","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53ea823d8ca6cef638c8df64: start","job_status":"start","queue_duration_s":2.185641,"scheduling_latency_s":2.185685}1751gitlab # [ 274.716342] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.510Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53ea823d8ca6cef638c8df64","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53ea823d8ca6cef638c8df64: done: 0.03731 sec","job_status":"done","queue_duration_s":2.185641,"scheduling_latency_s":2.185685,"redis_calls":13,"redis_duration_s":0.005398,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001798,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03731,"completed_at":"2026-09-01T10:36:09.510Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1752gitlab # [ 274.812823] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.513Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2718b823ffd4ef6c3a557bd","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2718b823ffd4ef6c3a557bd: start","job_status":"start","queue_duration_s":2.225565,"scheduling_latency_s":2.225603}1753gitlab # [ 274.814333] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.544Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2718b823ffd4ef6c3a557bd","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2718b823ffd4ef6c3a557bd: done: 0.03193 sec","job_status":"done","queue_duration_s":2.225565,"scheduling_latency_s":2.225603,"redis_calls":13,"redis_duration_s":0.004897,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001707,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00319,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020376,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03193,"completed_at":"2026-09-01T10:36:09.544Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1754gitlab # [ 274.838994] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.547Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1dfebb760cc4a94eda50a616","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1dfebb760cc4a94eda50a616: start","job_status":"start","queue_duration_s":2.260067,"scheduling_latency_s":2.260112}1755gitlab # [ 274.858057] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.580Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1dfebb760cc4a94eda50a616","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1dfebb760cc4a94eda50a616: done: 0.032914 sec","job_status":"done","queue_duration_s":2.260067,"scheduling_latency_s":2.260112,"redis_calls":13,"redis_duration_s":0.0053159999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001743,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003573,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020403,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032914,"completed_at":"2026-09-01T10:36:09.580Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1756gitlab # [ 274.927340] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9742ab7f373b947539580ffc","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9742ab7f373b947539580ffc: start","job_status":"start","queue_duration_s":2.2956,"scheduling_latency_s":2.295644}1757gitlab # [ 274.937320] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.607Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9742ab7f373b947539580ffc","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9742ab7f373b947539580ffc: done: 0.024605 sec","job_status":"done","queue_duration_s":2.2956,"scheduling_latency_s":2.295644,"redis_calls":13,"redis_duration_s":0.004008,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001194,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002814,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024605,"completed_at":"2026-09-01T10:36:09.607Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1758gitlab # [ 275.015759] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.609Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb323ccce084d7f81e51fb07","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb323ccce084d7f81e51fb07: start","job_status":"start","queue_duration_s":2.322202,"scheduling_latency_s":2.322239}1759gitlab # [ 275.019439] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.635Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb323ccce084d7f81e51fb07","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb323ccce084d7f81e51fb07: done: 0.025431 sec","job_status":"done","queue_duration_s":2.322202,"scheduling_latency_s":2.322239,"redis_calls":13,"redis_duration_s":0.00449,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001326,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003164,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016326,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025431,"completed_at":"2026-09-01T10:36:09.634Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1760gitlab # [ 275.133840] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.637Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef6277a46e0f2fb5a0e96804","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef6277a46e0f2fb5a0e96804: start","job_status":"start","queue_duration_s":2.350156,"scheduling_latency_s":2.350201}1761gitlab # [ 275.137688] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef6277a46e0f2fb5a0e96804","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef6277a46e0f2fb5a0e96804: done: 0.057586 sec","job_status":"done","queue_duration_s":2.350156,"scheduling_latency_s":2.350201,"redis_calls":13,"redis_duration_s":0.008082,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005479,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002603,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017379,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.057586,"completed_at":"2026-09-01T10:36:09.695Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1762gitlab # [ 275.202256] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.700Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"642ae06a59eaa926c92abb08","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-642ae06a59eaa926c92abb08: start","job_status":"start","queue_duration_s":2.412941,"scheduling_latency_s":2.412986}1763gitlab # [ 275.271451] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.894Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"642ae06a59eaa926c92abb08","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-642ae06a59eaa926c92abb08: done: 0.193893 sec","job_status":"done","queue_duration_s":2.412941,"scheduling_latency_s":2.412986,"redis_calls":13,"redis_duration_s":0.141986,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005865,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.136121,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023267,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.193893,"completed_at":"2026-09-01T10:36:09.894Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1764gitlab # [ 275.329922] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:09.898Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5322ed4f6f220027a1ed405e","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5322ed4f6f220027a1ed405e: start","job_status":"start","queue_duration_s":2.611366,"scheduling_latency_s":2.61141}1765gitlab # [ 275.349810] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:10.031Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5322ed4f6f220027a1ed405e","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5322ed4f6f220027a1ed405e: done: 0.132953 sec","job_status":"done","queue_duration_s":2.611366,"scheduling_latency_s":2.61141,"redis_calls":13,"redis_duration_s":0.045942000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.040902,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023596,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.132953,"completed_at":"2026-09-01T10:36:10.031Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1766gitlab # [ 275.351189] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:10.034Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0694ec39d40aa20ca74b402e","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0694ec39d40aa20ca74b402e: start","job_status":"start","queue_duration_s":2.74721,"scheduling_latency_s":2.747244}1767gitlab # [ 275.360914] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:10.094Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0694ec39d40aa20ca74b402e","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0694ec39d40aa20ca74b402e: done: 0.060329 sec","job_status":"done","queue_duration_s":2.74721,"scheduling_latency_s":2.747244,"redis_calls":13,"redis_duration_s":0.006796,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001396,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.060329,"completed_at":"2026-09-01T10:36:10.094Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1768gitlab # [ 275.452381] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:10.101Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91c4a78cfea76f8da4b38514","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91c4a78cfea76f8da4b38514: start","job_status":"start","queue_duration_s":2.813699,"scheduling_latency_s":2.813733}1769gitlab # [ 275.453784] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:10.160Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91c4a78cfea76f8da4b38514","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91c4a78cfea76f8da4b38514: done: 0.059832 sec","job_status":"done","queue_duration_s":2.813699,"scheduling_latency_s":2.813733,"redis_calls":13,"redis_duration_s":0.019715,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.006609,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.013106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017263,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.059832,"completed_at":"2026-09-01T10:36:10.160Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1770gitlab # [ 275.556471] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:10.164Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4af9a52d5b522185c44d9d7a","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4af9a52d5b522185c44d9d7a: start","job_status":"start","queue_duration_s":2.876548,"scheduling_latency_s":2.876593}1771gitlab # [ 275.557901] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:10.250Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4af9a52d5b522185c44d9d7a","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4af9a52d5b522185c44d9d7a: done: 0.086207 sec","job_status":"done","queue_duration_s":2.876548,"scheduling_latency_s":2.876593,"redis_calls":13,"redis_duration_s":0.017458,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00375,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.013708,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.086207,"completed_at":"2026-09-01T10:36:10.250Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1772gitlab # [ 275.627512] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:10.257Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec1026873e7b1f58b1b561fd","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec1026873e7b1f58b1b561fd: start","job_status":"start","queue_duration_s":2.969769,"scheduling_latency_s":2.969812}1773gitlab # [ 275.629255] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:10.318Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec1026873e7b1f58b1b561fd","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec1026873e7b1f58b1b561fd: done: 0.061274 sec","job_status":"done","queue_duration_s":2.969769,"scheduling_latency_s":2.969812,"redis_calls":13,"redis_duration_s":0.007451999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00226,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005192,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019189,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.061274,"completed_at":"2026-09-01T10:36:10.318Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1774gitlab # [ 275.697221] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:10.321Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d10be43faf75b7ed69d2837","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d10be43faf75b7ed69d2837: start","job_status":"start","queue_duration_s":3.034233,"scheduling_latency_s":3.034272}1775gitlab # [ 275.698471] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:10.374Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d10be43faf75b7ed69d2837","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d10be43faf75b7ed69d2837: done: 0.052451 sec","job_status":"done","queue_duration_s":3.034233,"scheduling_latency_s":3.034272,"redis_calls":13,"redis_duration_s":0.011514,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004834,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00668,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017469,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.052451,"completed_at":"2026-09-01T10:36:10.374Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1776gitlab # [ 275.786972] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:10.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fe9d4b8e64d829cef65ec3e","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fe9d4b8e64d829cef65ec3e: start","job_status":"start","queue_duration_s":3.090689,"scheduling_latency_s":3.090726}1777gitlab # [ 275.788261] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:10.470Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fe9d4b8e64d829cef65ec3e","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fe9d4b8e64d829cef65ec3e: done: 0.09223 sec","job_status":"done","queue_duration_s":3.090689,"scheduling_latency_s":3.090726,"redis_calls":13,"redis_duration_s":0.049052,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.012829,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.036223,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019503,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.09223,"completed_at":"2026-09-01T10:36:10.470Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1778gitlab # [ 275.861100] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:10.479Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b03802987a5cbf208fbd2021","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b03802987a5cbf208fbd2021: start","job_status":"start","queue_duration_s":3.192248,"scheduling_latency_s":3.192282}1779gitlab # [ 275.862339] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:10.578Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b03802987a5cbf208fbd2021","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b03802987a5cbf208fbd2021: done: 0.098642 sec","job_status":"done","queue_duration_s":3.192248,"scheduling_latency_s":3.192282,"redis_calls":13,"redis_duration_s":0.025364,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001253,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.024111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.098642,"completed_at":"2026-09-01T10:36:10.578Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1780gitlab # [ 275.946951] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:10.588Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"347385412a05844a5ed0a1b8","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-347385412a05844a5ed0a1b8: start","job_status":"start","queue_duration_s":3.300642,"scheduling_latency_s":3.300683}1781gitlab # [ 275.948472] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:10.642Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"347385412a05844a5ed0a1b8","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-347385412a05844a5ed0a1b8: done: 0.054147 sec","job_status":"done","queue_duration_s":3.300642,"scheduling_latency_s":3.300683,"redis_calls":13,"redis_duration_s":0.014971,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003795,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.011176,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017137,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.054147,"completed_at":"2026-09-01T10:36:10.642Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1782gitlab # [ 276.010544] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:10.646Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66dfac45247452f44135c92c","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66dfac45247452f44135c92c: start","job_status":"start","queue_duration_s":3.35947,"scheduling_latency_s":3.359512}1783gitlab # [ 276.012230] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:10.786Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66dfac45247452f44135c92c","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66dfac45247452f44135c92c: done: 0.139339 sec","job_status":"done","queue_duration_s":3.35947,"scheduling_latency_s":3.359512,"redis_calls":13,"redis_duration_s":0.043504,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.009447,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.034057,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020407,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.139339,"completed_at":"2026-09-01T10:36:10.786Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1784gitlab # [ 276.083142] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:10.790Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05d879ab335244e6ba0b9a47","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05d879ab335244e6ba0b9a47: start","job_status":"start","queue_duration_s":3.502657,"scheduling_latency_s":3.502695}1785gitlab # [ 276.084312] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:10.872Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05d879ab335244e6ba0b9a47","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05d879ab335244e6ba0b9a47: done: 0.082593 sec","job_status":"done","queue_duration_s":3.502657,"scheduling_latency_s":3.502695,"redis_calls":13,"redis_duration_s":0.036826,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00236,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.034466,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019463,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.082593,"completed_at":"2026-09-01T10:36:10.872Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1786gitlab # [ 276.155098] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:10.878Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"158cf4326b4592a9cdb78c17","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-158cf4326b4592a9cdb78c17: start","job_status":"start","queue_duration_s":3.590677,"scheduling_latency_s":3.590723}1787gitlab # [ 276.156535] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:10.949Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"158cf4326b4592a9cdb78c17","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-158cf4326b4592a9cdb78c17: done: 0.070972 sec","job_status":"done","queue_duration_s":3.590677,"scheduling_latency_s":3.590723,"redis_calls":13,"redis_duration_s":0.01775,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005579,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.012171,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019754,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.070972,"completed_at":"2026-09-01T10:36:10.948Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1788gitlab # [ 276.253949] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:11.853Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c7c7bd5e3af3e0a8129273a","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c7c7bd5e3af3e0a8129273a: start","job_status":"start","queue_duration_s":4.566266,"scheduling_latency_s":4.566314}1789gitlab # [ 276.540891] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:12.139Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c7c7bd5e3af3e0a8129273a","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c7c7bd5e3af3e0a8129273a: done: 0.28567 sec","job_status":"done","queue_duration_s":4.566266,"scheduling_latency_s":4.566314,"redis_calls":13,"redis_duration_s":0.024667,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.009297,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.01537,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022802,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.28567,"completed_at":"2026-09-01T10:36:12.139Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1790gitlab # [ 276.607936] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:12.207Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38cb753a92af551068a7a9f1","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38cb753a92af551068a7a9f1: start","job_status":"start","queue_duration_s":4.920078,"scheduling_latency_s":4.920125}1791gitlab # [ 276.672909] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:12.272Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38cb753a92af551068a7a9f1","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38cb753a92af551068a7a9f1: done: 0.065247 sec","job_status":"done","queue_duration_s":4.920078,"scheduling_latency_s":4.920125,"redis_calls":13,"redis_duration_s":0.026855,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.006859,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.019996,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.065247,"completed_at":"2026-09-01T10:36:12.272Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1792gitlab # [ 276.732344] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:12.279Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14a9499fdbb9efe178a4af0a","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14a9499fdbb9efe178a4af0a: start","job_status":"start","queue_duration_s":4.991833,"scheduling_latency_s":4.991878}1793gitlab # [ 276.809782] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:12.328Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14a9499fdbb9efe178a4af0a","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14a9499fdbb9efe178a4af0a: done: 0.049683 sec","job_status":"done","queue_duration_s":4.991833,"scheduling_latency_s":4.991878,"redis_calls":13,"redis_duration_s":0.010174,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002964,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00721,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018368,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.049683,"completed_at":"2026-09-01T10:36:12.328Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1794gitlab # [ 276.879403] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:12.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"80d4be15989fe51981b54ae7","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-80d4be15989fe51981b54ae7: start","job_status":"start","queue_duration_s":5.044713,"scheduling_latency_s":5.044757}1795gitlab # [ 276.881260] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:12.405Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"80d4be15989fe51981b54ae7","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-80d4be15989fe51981b54ae7: done: 0.073075 sec","job_status":"done","queue_duration_s":5.044713,"scheduling_latency_s":5.044757,"redis_calls":13,"redis_duration_s":0.031387,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.007866,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.023521,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.073075,"completed_at":"2026-09-01T10:36:12.405Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1796gitlab # [ 276.915449] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:12.416Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db5b4f044df7091476c0909b","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db5b4f044df7091476c0909b: start","job_status":"start","queue_duration_s":5.12949,"scheduling_latency_s":5.129534}1797gitlab # [ 276.950878] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:12.550Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db5b4f044df7091476c0909b","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db5b4f044df7091476c0909b: done: 0.13413 sec","job_status":"done","queue_duration_s":5.12949,"scheduling_latency_s":5.129534,"redis_calls":13,"redis_duration_s":0.10009799999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.081743,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.018355,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.13413,"completed_at":"2026-09-01T10:36:12.550Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1798gitlab # [ 277.028044] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:12.553Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"390a94507844359bf28808cc","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-390a94507844359bf28808cc: start","job_status":"start","queue_duration_s":5.265598,"scheduling_latency_s":5.265629}1799gitlab # [ 277.029119] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:12.578Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"390a94507844359bf28808cc","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-390a94507844359bf28808cc: done: 0.025012 sec","job_status":"done","queue_duration_s":5.265598,"scheduling_latency_s":5.265629,"redis_calls":13,"redis_duration_s":0.004416,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001155,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003261,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025012,"completed_at":"2026-09-01T10:36:12.577Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1800gitlab # [ 277.096975] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:12.580Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb283253a2072f2e92e209bd","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb283253a2072f2e92e209bd: start","job_status":"start","queue_duration_s":5.292956,"scheduling_latency_s":5.292993}1801gitlab # [ 277.098055] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:12.605Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb283253a2072f2e92e209bd","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb283253a2072f2e92e209bd: done: 0.025366 sec","job_status":"done","queue_duration_s":5.292956,"scheduling_latency_s":5.292993,"redis_calls":13,"redis_duration_s":0.0052380000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00121,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004028,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015882,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025366,"completed_at":"2026-09-01T10:36:12.605Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1802gitlab # [ 277.168184] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:12.607Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eaaa0535979b6d7874c512fe","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eaaa0535979b6d7874c512fe: start","job_status":"start","queue_duration_s":5.320189,"scheduling_latency_s":5.320215}1803gitlab # [ 277.169944] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:12.630Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eaaa0535979b6d7874c512fe","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eaaa0535979b6d7874c512fe: done: 0.022711 sec","job_status":"done","queue_duration_s":5.320189,"scheduling_latency_s":5.320215,"redis_calls":13,"redis_duration_s":0.003965,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001124,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002841,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022711,"completed_at":"2026-09-01T10:36:12.630Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1804gitlab # [ 277.266234] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:12.632Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4082d2a08e0fe5310ede4c0a","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4082d2a08e0fe5310ede4c0a: start","job_status":"start","queue_duration_s":5.34543,"scheduling_latency_s":5.345462}1805gitlab # [ 277.267311] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:12.655Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4082d2a08e0fe5310ede4c0a","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4082d2a08e0fe5310ede4c0a: done: 0.022436 sec","job_status":"done","queue_duration_s":5.34543,"scheduling_latency_s":5.345462,"redis_calls":13,"redis_duration_s":0.003737,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001466,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002271,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014211,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022436,"completed_at":"2026-09-01T10:36:12.655Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1806gitlab # [ 277.333370] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:12.657Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6de300fc5e389bd5745743d3","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6de300fc5e389bd5745743d3: start","job_status":"start","queue_duration_s":5.369969,"scheduling_latency_s":5.370002}1807gitlab # [ 277.334766] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:12.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6de300fc5e389bd5745743d3","created_at":"2026-09-01T10:36:06.965Z","trace_propagation_headers":{"sentry-trace":"539e65e96def4e98bdf6d2b378b3a479-9024a245c142404e","baggage":"sentry-trace_id=539e65e96def4e98bdf6d2b378b3a479,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0b804642346a1551d99c8d5c07854da1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:07.287Z","job_size_bytes":2,"pid":1811,"sidekiq_tid":"mamf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6de300fc5e389bd5745743d3: done: 0.021158 sec","job_status":"done","queue_duration_s":5.369969,"scheduling_latency_s":5.370002,"redis_calls":13,"redis_duration_s":0.003201,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000991,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00221,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013385,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021158,"completed_at":"2026-09-01T10:36:12.678Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1808gitlab # [ 293.922879] postgres[3054]: [3054] ERROR: relation "table_without_models" does not exist1809gitlab # [ 293.924758] postgres[3054]: [3054] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1810gitlab # [ 298.609247] runuser[2581]: pam_unix(runuser:session): session closed for user gitlab1811gitlab # [ 298.611302] setup-gitlab-start[2560]: + systemctl restart gitlab-sidekiq.service1812gitlab # [ 298.627651] systemd[1]: Stopping gitlab-sidekiq.service...1813gitlab # [ 298.629462] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:34.228Z","message":"Shutting down"}1814gitlab # [ 298.631459] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:34.229Z","message":"Scheduler exiting..."}1815gitlab # [ 298.633427] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:34.229Z","message":"Terminating quiet threads for default capsule"}1816gitlab # [ 298.635757] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:34.229Z","message":"Scheduler exiting..."}1817gitlab # [ 299.129923] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:34.730Z","message":"Pausing to allow jobs to finish..."}1818gitlab # [ 302.632907] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:38.233Z","message":"Bye!"}1819gitlab # [ 302.650064] sidekiq-cluster[1811]: {"severity":"INFO","time":"2026-09-01T10:36:38.250Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1811,"worker_id":"sidekiq_0","memwd_rss_bytes":944041984,"retry":0}1820gitlab # [ 303.553068] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1821gitlab # [ 303.554793] systemd[1]: Stopped gitlab-sidekiq.service.1822gitlab # [ 303.555800] systemd[1]: gitlab-sidekiq.service: Consumed 1min 37.727s CPU time over 3min 32.249s wall clock time, 955.4M memory peak, 4M read from disk.1823gitlab # [ 303.560836] systemd[1]: Started gitlab-sidekiq.service.1824gitlab # [ 303.563105] setup-gitlab-start[2560]: + mkdir -p /var/lib/secrets1825gitlab # [ 303.564250] setup-gitlab-start[2560]: + echo nixbot-test-token-of-20-chars1826gitlab # [ 303.564924] setup-gitlab-start[2560]: + chmod 644 /var/lib/secrets/gitlab-token1827gitlab # [ 303.566172] setup-gitlab-start[2560]: + curl -fs -X POST -H 'PRIVATE-TOKEN: nixbot-test-token-of-20-chars' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects1828gitlab # [ 304.403336] gitaly[932]: time="2026-09-01T10:36:40.003Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NPW7TVGQWHDPJQSKCS7P error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:36:49.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:40.003" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1829gitlab # [ 304.404843] gitaly[932]: time="2026-09-01T10:36:40.004Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NPW7TVGQWHDPJQSKCS7P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:36:49.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:40.003" grpc.time_ms=0.441 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1830gitlab # [ 304.552588] gitaly[932]: time="2026-09-01T10:36:40.153Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NPW7TVGQWHDPJQSKCS7P error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:36:50.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:40.152" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1831gitlab # [ 304.553542] gitaly[932]: time="2026-09-01T10:36:40.153Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NPW7TVGQWHDPJQSKCS7P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:36:50.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:40.152" grpc.time_ms=0.431 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1832gitlab # [ 304.603567] gitaly[932]: time="2026-09-01T10:36:40.204Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NPW7TVGQWHDPJQSKCS7P diskcache=9dd07817-3971-4eda-a89c-9b00a320f5ec grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-01T10:36:50.012" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:40.163" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1833gitlab # [ 304.604397] gitaly[932]: time="2026-09-01T10:36:40.204Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1572 command.oublock=32 command.real_time_ms=17 command.spawn_token_fork_ms=5 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NPW7TVGQWHDPJQSKCS7P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-01T10:36:50.012" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:40.163" grpc.time_ms=40.868 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1834gitlab # [ 304.620224] gitaly[932]: time="2026-09-01T10:36:40.220Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NPW7TVGQWHDPJQSKCS7P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:36:50.069" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:40.220" pid=932 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1835gitlab # [ 304.621479] gitaly[932]: time="2026-09-01T10:36:40.221Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NPW7TVGQWHDPJQSKCS7P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:36:50.069" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:40.220" grpc.time_ms=0.596 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1836gitlab # [ 304.634494] gitaly[932]: time="2026-09-01T10:36:40.235Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NPW7TVGQWHDPJQSKCS7P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-01T10:36:50.076" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:40.227" grpc.time_ms=6.797 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1837gitlab # [ 304.712767] gitaly[932]: time="2026-09-01T10:36:40.313Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NPW7TVGQWHDPJQSKCS7P error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:36:50.162" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:40.313" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1838gitlab # [ 304.728035] gitaly[932]: time="2026-09-01T10:36:40.328Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NPW7TVGQWHDPJQSKCS7P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:36:50.162" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:40.313" grpc.time_ms=0.294 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1839gitlab # [ 304.766386] gitaly[932]: time="2026-09-01T10:36:40.366Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NPW7TVGQWHDPJQSKCS7P diskcache=1b6c8863-4f6d-4efa-bbf9-388ecd3cdbdb grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-01T10:36:50.181" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:40.332" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1840gitlab # [ 304.776576] gitaly[932]: time="2026-09-01T10:36:40.377Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NPW7TVGQWHDPJQSKCS7P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:36:50.225" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:40.376" pid=932 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1841gitlab # [ 304.778664] gitaly[932]: time="2026-09-01T10:36:40.377Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NPW7TVGQWHDPJQSKCS7P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:36:50.225" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:40.376" grpc.time_ms=0.583 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1842gitlab # [ 304.814970] gitaly[932]: time="2026-09-01T10:36:40.415Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1569 command.oublock=32 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NPW7TVGQWHDPJQSKCS7P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-01T10:36:50.181" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:40.332" grpc.time_ms=34.863 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1843gitlab # [ 304.837777] gitaly[932]: time="2026-09-01T10:36:40.438Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=15 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NPW7TVGQWHDPJQSKCS7P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-01T10:36:50.266" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:40.417" grpc.time_ms=20.3 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1844gitlab # [ 304.893356] gitaly[932]: time="2026-09-01T10:36:40.493Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NPW7TVGQWHDPJQSKCS7P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:36:50.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:40.493" grpc.time_ms=0.208 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1845gitlab # [ 304.907168] gitaly[932]: time="2026-09-01T10:36:40.505Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NPW7TVGQWHDPJQSKCS7P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-01T10:36:50.013" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:40.504" grpc.time_ms=0.296 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1846gitlab # [ 305.550971] gitaly[932]: time="2026-09-01T10:36:41.151Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NPW7TVGQWHDPJQSKCS7P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-01T10:36:51.003" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:41.144" grpc.time_ms=6.4 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1847gitlab # [ 306.208979] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:41 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 26331848gitlab # [ 306.212413] setup-gitlab-start[2560]: + mkdir -p /tmp/test-flake1849gitlab # [ 306.214369] setup-gitlab-start[2560]: + cd /tmp/test-flake1850gitlab # [ 306.217572] setup-gitlab-start[2560]: + git init -b master1851gitlab # [ 306.229507] setup-gitlab-start[3157]: Initialized empty Git repository in /tmp/test-flake/.git/1852gitlab # [ 306.231495] setup-gitlab-start[2560]: + git config user.name test1853gitlab # [ 306.233355] setup-gitlab-start[2560]: + git config user.email test@example.com1854gitlab # [ 306.236457] setup-gitlab-start[2560]: + cat1855gitlab # [ 306.238770] setup-gitlab-start[2560]: + git add flake.nix1856gitlab # [ 306.241118] setup-gitlab-start[2560]: + git commit -m 'initial commit'1857gitlab # [ 306.247921] setup-gitlab-start[3162]: [master (root-commit) 7bdb102] initial commit1858gitlab # [ 306.249633] setup-gitlab-start[3162]: 1 file changed, 10 insertions(+)1859gitlab # [ 306.251287] setup-gitlab-start[3162]: create mode 100644 flake.nix1860gitlab # [ 306.252887] setup-gitlab-start[2560]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1861gitlab # [ 306.256241] setup-gitlab-start[2560]: + git push -u origin master1862gitlab # [ 306.312379] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:41 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 351863gitlab # [ 306.955129] gitaly[932]: time="2026-09-01T10:36:42.555Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NSHYF7G4NHVFT372CVMP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:42.547" grpc.time_ms=7.393 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1864gitlab # [ 306.976585] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:42 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 6581865gitlab # [ 307.229441] gitaly[932]: time="2026-09-01T10:36:42.829Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-01T10:36:52.010" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:42.822" grpc.time_ms=7.58 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1866gitlab # [ 307.283643] gitaly[932]: time="2026-09-01T10:36:42.884Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=330 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-01T10:37:12.077" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-01T10:36:42.878" grpc.time_ms=5.92 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1867gitlab # [ 307.331301] gitaly[932]: time="2026-09-01T10:36:42.931Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB duration_ms=204ns method=POST pid=932 status=200 url="http://unix/api/v4/internal/allowed"1868gitlab # [ 307.431335] redis[795]: 10 changes in 300 seconds. Saving...1869gitlab # [ 307.432201] redis[795]: Background saving started by pid 32051870gitlab # [ 307.439379] redis[3205]: BGSAVE done, 449 keys saved, 0 keys skipped, 113673 bytes written.1871gitlab # [ 307.444864] redis[3205]: DB saved on disk1872gitlab # [ 307.446643] redis[3205]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB1873gitlab # [ 307.469440] gitaly[932]: time="2026-09-01T10:36:43.069Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB duration_ms=133ns method=POST pid=932 status=200 url="http://unix/api/v4/internal/pre_receive"1874gitlab # [ 307.470093] gitaly[932]: time="2026-09-01T10:36:43.070Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:42.726" grpc.time_ms=343.779 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1875gitlab # [ 307.513182] gitaly[932]: time="2026-09-01T10:36:43.113Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:43.112" grpc.time_ms=1.084 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1876gitlab # [ 307.532410] redis[795]: Background saving terminated with success1877gitlab # [ 307.558798] gitaly[932]: time="2026-09-01T10:36:43.159Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:43.158" grpc.time_ms=0.54 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1878gitlab # [ 307.599471] gitaly[932]: time="2026-09-01T10:36:43.199Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:43.198" grpc.time_ms=1.599 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1879gitlab # [ 307.783790] gitaly[932]: time="2026-09-01T10:36:43.383Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-01T10:36:53.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:43.376" grpc.time_ms=6.619 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1880gitlab # [ 307.809523] gitaly[932]: time="2026-09-01T10:36:43.409Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1563 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-01T10:36:53.023" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:36:43.394" grpc.time_ms=15.374 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1881gitlab # [ 307.830244] gitaly[932]: time="2026-09-01T10:36:43.430Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB duration_ms=190ns method=POST pid=932 status=200 url="http://unix/api/v4/internal/post_receive"1882gitlab # [ 307.835813] gitaly[932]: time="2026-09-01T10:36:43.431Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:43.239" grpc.time_ms=192.224 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1883gitlab # [ 307.850470] gitaly[932]: time="2026-09-01T10:36:43.449Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB diskcache=c2957467-9c6c-46b2-9a52-5211ae56f1f2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:42.668" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1884gitlab # [ 307.851084] gitlab-workhorse[832]: time="2026-09-01T10:36:43Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521885gitlab # [ 307.851233] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:43 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 8621886gitlab # [ 307.851429] gitaly[932]: time="2026-09-01T10:36:43.449Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=180 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=14110 command.oublock=32 command.real_time_ms=765 command.spawn_token_fork_ms=0 command.system_time_ms=106 command.user_time_ms=74 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:42.668" grpc.time_ms=780.968 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1887gitlab # [ 307.888694] setup-gitlab-start[3166]: To http://gitlab/root/test-flake.git1888gitlab # [ 307.890323] setup-gitlab-start[3166]: * [new branch] master -> master1889gitlab # [ 307.894349] setup-gitlab-start[3166]: branch 'master' set up to track 'origin/master'.1890gitlab # [ 307.897491] systemd[1]: Finished setup-gitlab.service.1891gitlab # [ 307.901160] systemd[1]: Starting nixbot CI service...1892gitlab: (finished: waiting for unit setup-gitlab.service, in 308.92 seconds)1893gitlab: waiting for unit nixbot.service1894gitlab # [ 308.862910] postgres[864]: [864] LOG: checkpoint starting: time1895gitlab # [ 309.730066] gitaly[932]: time="2026-09-01T10:36:45.330Z" level=info msg="burden monitor stats" pid=9321896gitlab # [ 309.797437] sidekiq-cluster[3104]: {"severity":"INFO","time":"2026-09-01T10:36:45.355Z","message":"Starting cluster with 1 processes"}1897gitlab # [ 310.346058] nixbot[3255]: {"ts": "2026-09-01T10:36:45.946Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1898gitlab # [ 310.378646] nixbot[3255]: {"ts": "2026-09-01T10:36:45.979Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1899gitlab # [ 310.407905] nixbot[3255]: {"ts": "2026-09-01T10:36:46.008Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1900gitlab # [ 310.417058] nixbot[3255]: {"ts": "2026-09-01T10:36:46.017Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1901gitlab # [ 310.426457] nixbot[3255]: {"ts": "2026-09-01T10:36:46.026Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1902gitlab # [ 310.431937] nixbot[3255]: {"ts": "2026-09-01T10:36:46.032Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1903gitlab # [ 310.439447] nixbot[3255]: {"ts": "2026-09-01T10:36:46.039Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1904gitlab # [ 310.445398] nixbot[3255]: {"ts": "2026-09-01T10:36:46.045Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1905gitlab # [ 310.453439] nixbot[3255]: {"ts": "2026-09-01T10:36:46.053Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1906gitlab # [ 310.458813] nixbot[3255]: {"ts": "2026-09-01T10:36:46.059Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1907gitlab # [ 310.464132] nixbot[3255]: {"ts": "2026-09-01T10:36:46.064Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1908gitlab # [ 310.474461] nixbot[3255]: {"ts": "2026-09-01T10:36:46.074Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1909gitlab # [ 310.481113] nixbot[3255]: {"ts": "2026-09-01T10:36:46.081Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1910gitlab # [ 310.486717] nixbot[3255]: {"ts": "2026-09-01T10:36:46.087Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1911gitlab # [ 310.493574] nixbot[3255]: {"ts": "2026-09-01T10:36:46.094Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1912gitlab # [ 310.496646] nixbot[3255]: {"ts": "2026-09-01T10:36:46.097Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1913gitlab # [ 310.503243] nixbot[3255]: {"ts": "2026-09-01T10:36:46.103Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1914gitlab # [ 310.509252] nixbot[3255]: {"ts": "2026-09-01T10:36:46.109Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1915gitlab # [ 310.518051] nixbot[3255]: {"ts": "2026-09-01T10:36:46.118Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1916gitlab # [ 310.521961] nixbot[3255]: {"ts": "2026-09-01T10:36:46.122Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1917gitlab # [ 310.528725] nixbot[3255]: {"ts": "2026-09-01T10:36:46.128Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1918gitlab # [ 310.535207] nixbot[3255]: {"ts": "2026-09-01T10:36:46.135Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1919gitlab # [ 310.541494] nixbot[3255]: {"ts": "2026-09-01T10:36:46.141Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1920gitlab # [ 310.546784] nixbot[3255]: {"ts": "2026-09-01T10:36:46.147Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1921gitlab # [ 310.552464] nixbot[3255]: {"ts": "2026-09-01T10:36:46.152Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1922gitlab # [ 310.561604] nixbot[3255]: {"ts": "2026-09-01T10:36:46.162Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1923gitlab # [ 310.567807] nixbot[3255]: {"ts": "2026-09-01T10:36:46.168Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1924gitlab # [ 310.575136] nixbot[3255]: {"ts": "2026-09-01T10:36:46.175Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1925gitlab # [ 311.099906] nixbot[3255]: INFO: Started server process [3255]1926gitlab # [ 311.100372] nixbot[3255]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1927gitlab # [ 311.130621] nixbot[3255]: INFO: ::ffff:127.0.0.1:44356 - "GET /health HTTP/1.1" 200 OK1928gitlab # [ 311.144279] systemd[1]: Started nixbot CI service.1929gitlab # [ 311.146527] systemd[1]: Reached target Multi-User System.1930gitlab # [ 311.153040] systemd[1]: Startup finished in 1.128s (kernel) + 3.250s (initrd) + 5min 6.772s (userspace) = 5min 11.151s.1931gitlab: (finished: waiting for unit nixbot.service, in 3.10 seconds)1932gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1933gitlab # [ 311.248157] nixbot[3255]: INFO: ::ffff:127.0.0.1:44362 - "GET /health HTTP/1.1" 200 OK1934gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.03 seconds)1935subtest: project discovered and webhook registered1936gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1937gitlab # [ 311.367648] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:46 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 881938gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.13 seconds)1939[]1940gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1941gitlab # [ 312.523511] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:48 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 791942gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.16 seconds)1943[]1944gitlab # [ 312.716880] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:48 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 15871945gitlab # [ 312.742079] nixbot[3255]: {"ts": "2026-09-01T10:36:48.341Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1946gitlab # [ 312.779568] nixbot[3255]: {"ts": "2026-09-01T10:36:48.379Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1947gitlab # [ 312.879312] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:48 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 761948gitlab # [ 312.880910] nixbot[3255]: {"ts": "2026-09-01T10:36:48.481Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1949gitlab # [ 312.882571] nixbot[3255]: {"ts": "2026-09-01T10:36:48.483Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1950gitlab # [ 313.085945] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:48 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 919 "" "python-httpx/0.28.1" 1981951gitlab # [ 313.089540] nixbot[3255]: {"ts": "2026-09-01T10:36:48.687Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1952gitlab # [ 313.358233] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:48 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 2531953gitlab # [ 313.362429] nixbot[3255]: {"ts": "2026-09-01T10:36:48.960Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1954gitlab # [ 313.481552] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 882 "" "python-httpx/0.28.1" 1051955gitlab # [ 313.483105] nixbot[3255]: {"ts": "2026-09-01T10:36:49.083Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1956gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1957gitlab # [ 313.591706] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:49 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 919 "" "python-httpx/0.28.1" 1031958gitlab # [ 313.593371] nixbot[3255]: {"ts": "2026-09-01T10:36:49.192Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1959gitlab # [ 313.691197] gitaly[932]: time="2026-09-01T10:36:49.291Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P0NFGAJT1RCJEWGYAAS6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-01T10:37:19.078" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:36:49.279" grpc.time_ms=11.315 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1960gitlab # [ 313.728745] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 882 "" "curl/8.21.0" 1601961gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.19 seconds)1962[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-01T10:36:48.656Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]1963(finished: subtest: project discovered and webhook registered, in 2.47 seconds)1964subtest: push delivers the webhook1965gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1966gitlab # [ 313.738039] gitaly[932]: time="2026-09-01T10:36:49.338Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P0NFGAJT1RCJEWGYAAS6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:37:19.137" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:36:49.337" grpc.time_ms=0.703 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1967gitlab # [ 313.763485] gitaly[932]: time="2026-09-01T10:36:49.364Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=6 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P0NFGAJT1RCJEWGYAAS6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-01T10:36:59.010" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:36:49.351" grpc.time_ms=12.123 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1968gitlab # [ 313.837063] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 2371969gitlab # [ 313.841782] nixbot[3255]: {"ts": "2026-09-01T10:36:49.438Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1970gitlab # [ 313.861760] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:49 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 391971gitlab # [ 313.978783] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1321972gitlab # [ 313.983801] nixbot[3255]: {"ts": "2026-09-01T10:36:49.584Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1973gitlab # [ 313.988534] nixbot[3255]: {"ts": "2026-09-01T10:36:49.588Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1974gitlab # [ 314.042176] gitaly[932]: time="2026-09-01T10:36:49.642Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=5 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8P0XXQAC8NBRBA9E9KMM8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:49.626" grpc.time_ms=15.366 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1975gitlab # [ 314.062748] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:49 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.55.0" 1781976gitlab # Enumerating objects: 5, done.1977gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1978gitlab # Delta compression using up to 4 threads1979gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1980gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 272 bytes | 136.00 KiB/s, done.1981gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1982gitlab # [ 314.143698] gitaly[932]: time="2026-09-01T10:36:49.744Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8P133GXV5N753W5YEN0XK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:49.725" grpc.time_ms=18.216 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc1983gitlab # [ 314.162565] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:49 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1091984gitlab # [ 314.535218] gitaly[932]: time="2026-09-01T10:36:50.135Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P16KH15V9BRRGFJGECVX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:50.122" pid=932 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1985gitlab # [ 314.535980] gitaly[932]: time="2026-09-01T10:36:50.135Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=6 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P16KH15V9BRRGFJGECVX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:50.122" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc1986gitlab # [ 314.537641] gitaly[932]: time="2026-09-01T10:36:50.138Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=6 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P16KH15V9BRRGFJGECVX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:50.122" grpc.time_ms=15.181 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc1987gitlab # [ 314.541383] gitlab-workhorse[832]: time="2026-09-01T10:36:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131988gitlab # [ 314.542974] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:50 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 3911989gitlab # [ 314.606915] gitaly[932]: time="2026-09-01T10:36:50.207Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=331 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=71 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-01T10:37:20.032" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-01T10:36:50.133" grpc.time_ms=73.952 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1990gitlab # [ 314.666519] gitaly[932]: time="2026-09-01T10:36:50.267Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z duration_ms=330ns method=POST pid=932 status=200 url="http://unix/api/v4/internal/allowed"1991gitlab # [ 314.688150] gitaly[932]: time="2026-09-01T10:36:50.286Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P1JZBVF2QV9741XJ5P8W grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:50.258" pid=932 remote_ip="2001:db8:1::1" request_sha=9acd99d972273a6944d1e7cda53c501381ca70bb response_bytes=223 span.kind=server system=grpc1992gitlab # [ 314.689188] gitlab-workhorse[832]: time="2026-09-01T10:36:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2231993gitlab # [ 314.690271] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:50 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 1441994gitlab # [ 314.692445] gitaly[932]: time="2026-09-01T10:36:50.286Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1207 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=4 command.system_time_ms=4 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P1JZBVF2QV9741XJ5P8W grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:50.258" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc1995gitlab # [ 314.695548] gitaly[932]: time="2026-09-01T10:36:50.287Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1207 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=4 command.system_time_ms=4 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P1JZBVF2QV9741XJ5P8W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:50.258" grpc.time_ms=28.563 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc1996gitlab # [ 314.781406] gitaly[932]: time="2026-09-01T10:36:50.381Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z duration_ms=113ns method=POST pid=932 status=200 url="http://unix/api/v4/internal/pre_receive"1997gitlab # [ 314.781869] gitaly[932]: time="2026-09-01T10:36:50.382Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:49.935" grpc.time_ms=446.77 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1998gitlab # [ 314.842649] gitaly[932]: time="2026-09-01T10:36:50.443Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:50.442" grpc.time_ms=0.604 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1999gitlab # [ 314.920689] gitaly[932]: time="2026-09-01T10:36:50.521Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:50.520" grpc.time_ms=0.86 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root2000gitlab # [ 314.941646] gitaly[932]: time="2026-09-01T10:36:50.542Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8P1W0D0TEMH12XQQC2AFP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:50.527" grpc.time_ms=14.841 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2001gitlab # [ 314.943135] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:50 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1102002gitlab # [ 315.003174] gitaly[932]: time="2026-09-01T10:36:50.603Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:50.602" grpc.time_ms=1.189 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root2003gitlab # [ 315.041573] gitaly[932]: time="2026-09-01T10:36:50.642Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P1ZHDZ134JCDVRZWPVWT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:50.623" pid=932 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2004gitlab # [ 315.042851] gitaly[932]: time="2026-09-01T10:36:50.642Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P1ZHDZ134JCDVRZWPVWT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:50.623" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2005gitlab # [ 315.043839] gitlab-workhorse[832]: time="2026-09-01T10:36:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672006gitlab # [ 315.044735] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:50 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 982007gitlab # [ 315.046469] gitaly[932]: time="2026-09-01T10:36:50.647Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P1ZHDZ134JCDVRZWPVWT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:50.623" grpc.time_ms=19.174 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2008gitlab # [ 315.205088] gitaly[932]: time="2026-09-01T10:36:50.805Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P22QYE4FC26NF0FCQNDZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:50.780" pid=932 remote_ip="2001:db8:1::1" request_sha=353c24fd0927ceafedb16734b4a351222e5489cc response_bytes=341 span.kind=server system=grpc2009gitlab # [ 315.205666] gitaly[932]: time="2026-09-01T10:36:50.805Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1187 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P22QYE4FC26NF0FCQNDZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:50.780" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2010gitlab # [ 315.206793] gitaly[932]: time="2026-09-01T10:36:50.806Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1187 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P22QYE4FC26NF0FCQNDZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:50.780" grpc.time_ms=25.054 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2011gitlab # [ 315.209796] gitlab-workhorse[832]: time="2026-09-01T10:36:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3412012gitlab # [ 315.220472] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:50 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 341 "" "git/2.55.0" 1612013gitlab # [ 315.277188] gitaly[932]: time="2026-09-01T10:36:50.875Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z duration_ms=160ns method=POST pid=932 status=200 url="http://unix/api/v4/internal/post_receive"2014gitlab # [ 315.282656] gitaly[932]: time="2026-09-01T10:36:50.875Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:50.713" grpc.time_ms=161.701 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root2015gitlab # [ 315.330461] gitaly[932]: time="2026-09-01T10:36:50.931Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z diskcache=8ef179eb-3b98-4983-a6f9-5a2598902b19 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:49.864" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root2016gitlab # [ 315.331762] gitaly[932]: time="2026-09-01T10:36:50.932Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=271 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13468 command.oublock=32 command.real_time_ms=1018 command.spawn_token_fork_ms=1 command.system_time_ms=188 command.user_time_ms=83 component=gitaly.StreamServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=604 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=58 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:49.864" grpc.time_ms=1067.758 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root2017gitlab # [ 315.333095] gitlab-workhorse[832]: time="2026-09-01T10:36:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=522018gitlab # [ 315.334050] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:50 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 12412019gitlab # To http://gitlab/root/test-flake.git2020gitlab # 7bdb102..e8d9360 master -> master2021gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.65 seconds)2022gitlab: must succeed: git -C /tmp/test-flake rev-parse master2023gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)2024gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2025gitlab # [ 315.502154] gitaly[932]: time="2026-09-01T10:36:51.102Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=5 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8P2E7P8TYD22ZDW8VP0TY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:51.087" grpc.time_ms=14.564 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2026gitlab # [ 315.503703] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:51 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 882027gitlab # [ 315.513313] nixbot[3255]: INFO: ::ffff:127.0.0.1:46836 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2028gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.14 seconds)2029[]2030gitlab # [ 315.649163] gitaly[932]: time="2026-09-01T10:36:51.249Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P2HX31EX7S3RZ7VCDNEJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:51.220" pid=932 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc2031gitlab # [ 315.650081] gitaly[932]: time="2026-09-01T10:36:51.249Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1169 command.oublock=0 command.real_time_ms=25 command.spawn_token_fork_ms=4 command.system_time_ms=14 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P2HX31EX7S3RZ7VCDNEJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:51.220" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2032gitlab # [ 315.650270] gitaly[932]: time="2026-09-01T10:36:51.249Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1169 command.oublock=0 command.real_time_ms=25 command.spawn_token_fork_ms=4 command.system_time_ms=14 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P2HX31EX7S3RZ7VCDNEJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:36:51.220" grpc.time_ms=28.994 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2033gitlab # [ 315.709925] gitlab-workhorse[832]: time="2026-09-01T10:36:51Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2092034gitlab # [ 315.710477] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:51 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 1772035gitlab # [ 315.881945] gitaly[932]: time="2026-09-01T10:36:51.482Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P2T0XK1F4TPTHNEXVHP1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:37:21.280" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:36:51.481" grpc.time_ms=0.869 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2036gitlab # [ 315.960246] gitaly[932]: time="2026-09-01T10:36:51.560Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8P2T0XK1F4TPTHNEXVHP1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-01T10:37:21.350" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:36:51.551" grpc.time_ms=8.993 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2037gitlab # [ 316.047699] gitaly[932]: time="2026-09-01T10:36:51.648Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8P2T0XK1F4TPTHNEXVHP1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:37:01.261" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:36:51.631" grpc.time_ms=12.504 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2038gitlab # [ 316.117915] gitaly[932]: time="2026-09-01T10:36:51.718Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8P2T0XK1F4TPTHNEXVHP1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:37:01.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:36:51.701" grpc.time_ms=16.861 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2039gitlab # [ 316.248911] gitaly[932]: time="2026-09-01T10:36:51.849Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P2T0XK1F4TPTHNEXVHP1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-01T10:37:01.157" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:51.848" grpc.time_ms=0.421 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2040gitlab # [ 316.291300] gitaly[932]: time="2026-09-01T10:36:51.891Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=620 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=8 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P2T0XK1F4TPTHNEXVHP1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-01T10:37:01.185" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:36:51.876" grpc.time_ms=14.273 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2041gitlab # [ 316.395457] gitaly[932]: time="2026-09-01T10:36:51.996Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8P2T0XK1F4TPTHNEXVHP1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:51.995" grpc.time_ms=0.443 pid=932 span.kind=server system=grpc2042gitlab # [ 316.451429] gitaly[932]: time="2026-09-01T10:36:52.051Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8P2T0XK1F4TPTHNEXVHP1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:36:52.051" grpc.time_ms=0.527 pid=932 span.kind=server system=grpc2043gitlab # [ 316.504714] gitaly[932]: time="2026-09-01T10:36:52.103Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P2T0XK1F4TPTHNEXVHP1 diskcache=32908f04-64b6-48d6-a9b2-94be0675e86e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-01T10:37:01.232" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:51.923" pid=932 ref=refs/keep-around/7bdb1025a7f20513f1ec42f289b6f901a9fe07ea remote_ip="2001:db8:1::1" revision=7bdb1025a7f20513f1ec42f289b6f901a9fe07ea span.kind=server system=grpc username=root2044gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2045gitlab # [ 316.505274] gitaly[932]: time="2026-09-01T10:36:52.104Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=84 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5429 command.oublock=8 command.real_time_ms=157 command.spawn_token_fork_ms=1 command.system_time_ms=51 command.user_time_ms=33 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P2T0XK1F4TPTHNEXVHP1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-01T10:37:01.232" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:36:51.923" grpc.time_ms=180.448 pid=932 ref=refs/keep-around/7bdb1025a7f20513f1ec42f289b6f901a9fe07ea remote_ip="2001:db8:1::1" revision=7bdb1025a7f20513f1ec42f289b6f901a9fe07ea span.kind=server system=grpc username=root2046gitlab # [ 316.512134] gitaly[932]: time="2026-09-01T10:36:52.112Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P2T0XK1F4TPTHNEXVHP1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:37:22.010" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:36:52.111" grpc.time_ms=0.656 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2047gitlab # [ 316.559513] nixbot[3255]: INFO: ::ffff:127.0.0.1:46844 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2048gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.04 seconds)2049[]2050gitlab # [ 316.741578] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:36:52 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7bdb1025a7f20513f1ec42f289b6f901a9fe07ea HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 9482051gitlab # [ 316.750064] nixbot[3255]: {"ts": "2026-09-01T10:36:52.350Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/7bdb1025a7f20513f1ec42f289b6f901a9fe07ea \"HTTP/1.1 201 Created\""}2052gitlab # [ 316.775377] nixbot[3255]: {"ts": "2026-09-01T10:36:52.375Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-7bdb1025a7f2-0"}2053gitlab # [ 317.019926] nixbot[3255]: {"ts": "2026-09-01T10:36:52.618Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-7bdb1025a7f2-0", "argv0": "bwrap"}2054gitlab # [ 317.153294] systemd[1]: Started Nix Daemon.2055gitlab # [ 317.216554] nix-daemon[3502]: accepted connection from pid 3497, user nixbot2056gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2057gitlab # [ 317.601095] nixbot[3255]: INFO: ::ffff:127.0.0.1:46860 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2058gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.03 seconds)2059[]2060gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2061gitlab # [ 318.633191] nixbot[3255]: INFO: ::ffff:127.0.0.1:46870 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2062gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.03 seconds)2063[]2064gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2065gitlab # [ 319.656140] nixbot[3255]: INFO: ::ffff:127.0.0.1:46872 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2066gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2067[]2068gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2069gitlab # [ 320.680032] nixbot[3255]: INFO: ::ffff:127.0.0.1:46878 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2070gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2071[]2072gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2073gitlab # [ 321.699352] nixbot[3255]: INFO: ::ffff:127.0.0.1:46882 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2074gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2075[]2076gitlab # [ 321.954816] nix-daemon[3502]: accepted connection from pid 3528, user nixbot2077gitlab # [ 321.965305] nix-daemon[3502]: accepted connection from pid 3497, user nixbot2078gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2079gitlab # [ 322.721890] nixbot[3255]: INFO: ::ffff:127.0.0.1:46888 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2080gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2081[]2082gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2083gitlab # [ 323.742984] nixbot[3255]: INFO: ::ffff:127.0.0.1:57784 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2084gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2085[]2086gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2087gitlab # [ 324.763671] nixbot[3255]: INFO: ::ffff:127.0.0.1:57786 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2088gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2089[]2090gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2091gitlab # [ 325.783723] nixbot[3255]: INFO: ::ffff:127.0.0.1:57796 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2092gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2093[]2094gitlab # [ 326.134446] nix-daemon[3502]: accepted connection from pid 3497, user nixbot2095gitlab # [ 326.191128] nix-daemon[3502]: accepted connection from pid 3554, user nixbot2096gitlab # [ 326.232425] gitaly[932]: time="2026-09-01T10:37:01.832Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8PCYK8J7TA4T8RVNCEKHT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:37:31.031" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:01.832" grpc.time_ms=0.503 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2097gitlab # [ 326.394806] gitaly[932]: time="2026-09-01T10:37:01.995Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8PCYK8J7TA4T8RVNCEKHT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:37:11.178" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:01.989" grpc.time_ms=6.009 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2098gitlab # [ 326.500335] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:02 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7bdb1025a7f20513f1ec42f289b6f901a9fe07ea HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 3202099gitlab # [ 326.506162] nixbot[3255]: {"ts": "2026-09-01T10:37:02.101Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/7bdb1025a7f20513f1ec42f289b6f901a9fe07ea \"HTTP/1.1 201 Created\""}2100gitlab # [ 326.553223] gitaly[932]: time="2026-09-01T10:37:02.153Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8PD8SXC2QPW0H78S5M7Z5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:37:32.052" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:02.152" grpc.time_ms=0.647 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2101gitlab # [ 326.625297] gitaly[932]: time="2026-09-01T10:37:02.225Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1E8PD8SXC2QPW0H78S5M7Z5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:37:12.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:02.211" grpc.time_ms=13.876 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2102gitlab # [ 326.729361] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:02 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7bdb1025a7f20513f1ec42f289b6f901a9fe07ea HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2232103gitlab # [ 326.732841] nixbot[3255]: {"ts": "2026-09-01T10:37:02.330Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/7bdb1025a7f20513f1ec42f289b6f901a9fe07ea \"HTTP/1.1 201 Created\""}2104gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2105gitlab # [ 326.808074] nixbot[3255]: INFO: ::ffff:127.0.0.1:57808 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2106gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2107[]2108gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2109gitlab # [ 327.828426] nixbot[3255]: INFO: ::ffff:127.0.0.1:57818 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2110gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2111[]2112gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2113gitlab # [ 328.848244] nixbot[3255]: INFO: ::ffff:127.0.0.1:57834 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2114gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2115[]2116gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2117gitlab # [ 329.868179] nixbot[3255]: INFO: ::ffff:127.0.0.1:57844 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2118gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2119[]2120gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2121gitlab # [ 330.887566] nixbot[3255]: INFO: ::ffff:127.0.0.1:57856 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2122gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2123[]2124gitlab # [ 331.333701] nix-daemon[3502]: accepted connection from pid 3604, user nixbot2125gitlab # [ 331.395237] gitaly[932]: time="2026-09-01T10:37:06.995Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8PJ0ACHS5AVZ5XTKFEC8D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:37:36.794" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:06.994" grpc.time_ms=0.54 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2126gitlab # [ 331.422885] gitaly[932]: time="2026-09-01T10:37:07.023Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8PJ0ACHS5AVZ5XTKFEC8D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:37:17.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:07.019" grpc.time_ms=4.344 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2127gitlab # [ 331.474413] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:07 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7bdb1025a7f20513f1ec42f289b6f901a9fe07ea HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1202128gitlab # [ 331.478064] nixbot[3255]: {"ts": "2026-09-01T10:37:07.075Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/7bdb1025a7f20513f1ec42f289b6f901a9fe07ea \"HTTP/1.1 201 Created\""}2129gitlab # [ 331.597077] gitaly[932]: time="2026-09-01T10:37:07.197Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8PJ6K6AJQJ0AMNE4QBZWY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:37:07.192" grpc.time_ms=4.651 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2130gitlab # [ 331.613678] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:07 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 422131gitlab # [ 331.644625] gitaly[932]: time="2026-09-01T10:37:07.245Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8PJ7Z2ZTARW01DS00ZFPE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:37:07.240" pid=932 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2132gitlab # [ 331.658903] gitaly[932]: time="2026-09-01T10:37:07.245Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8PJ7Z2ZTARW01DS00ZFPE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:37:07.240" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2133gitlab # [ 331.676817] gitlab-workhorse[832]: time="2026-09-01T10:37:07Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672134gitlab # [ 331.677591] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:37:07 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 462135gitlab # [ 331.678718] gitaly[932]: time="2026-09-01T10:37:07.245Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8PJ7Z2ZTARW01DS00ZFPE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:37:07.240" grpc.time_ms=5.342 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2136gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2137gitlab # [ 331.908248] nixbot[3255]: INFO: ::ffff:127.0.0.1:57872 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2138gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2139[]2140gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2141gitlab # [ 332.974715] nixbot[3255]: INFO: ::ffff:127.0.0.1:57880 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2142gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.07 seconds)2143[]2144gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2145gitlab # [ 333.994662] nixbot[3255]: INFO: ::ffff:127.0.0.1:55276 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2146gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2147[]2148gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2149gitlab # [ 335.015932] nixbot[3255]: INFO: ::ffff:127.0.0.1:55284 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2150gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2151[]2152gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2153gitlab # [ 336.035580] nixbot[3255]: INFO: ::ffff:127.0.0.1:55298 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2154gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2155[]2156gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2157gitlab # [ 337.056976] nixbot[3255]: INFO: ::ffff:127.0.0.1:55304 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2158gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2159[]2160gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2161gitlab # [ 338.074947] nixbot[3255]: INFO: ::ffff:127.0.0.1:55316 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2162gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2163[]2164gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2165gitlab # [ 339.092863] nixbot[3255]: INFO: ::ffff:127.0.0.1:55320 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2166gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2167[]2168gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2169gitlab # [ 340.116253] nixbot[3255]: INFO: ::ffff:127.0.0.1:55324 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2170gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2171[]2172gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2173gitlab # [ 341.138184] nixbot[3255]: INFO: ::ffff:127.0.0.1:55328 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2174gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2175[]2176gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2177gitlab # [ 342.158545] nixbot[3255]: INFO: ::ffff:127.0.0.1:55344 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2178gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2179[]2180gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2181gitlab # [ 343.177640] nixbot[3255]: INFO: ::ffff:127.0.0.1:41550 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2182gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2183[]2184gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2185gitlab # [ 344.197658] nixbot[3255]: INFO: ::ffff:127.0.0.1:41552 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2186gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2187[]2188gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2189gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2190[]2191gitlab # [ 345.215820] nixbot[3255]: INFO: ::ffff:127.0.0.1:41568 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2192gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2193gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2194[]2195gitlab # [ 346.233652] nixbot[3255]: INFO: ::ffff:127.0.0.1:41578 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2196gitlab # [ 346.805939] sidekiq-cluster[3276]: 2026-09-01T10:37:22.406Z pid=3276 tid=yg4 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3276, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>433827840}2197gitlab # [ 347.053089] sidekiq-cluster[3276]: W, [2026-09-01T10:37:22.653544 #3276] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.2198gitlab # [ 347.102540] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:22.702Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}2199gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2200gitlab # [ 347.253129] nixbot[3255]: INFO: ::ffff:127.0.0.1:41586 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2201gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2202[]2203gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2204gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2205[]2206gitlab # [ 348.273506] nixbot[3255]: INFO: ::ffff:127.0.0.1:41600 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2207gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2208gitlab # [ 349.274582] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:24.874Z","message":"GitLab reliable fetch activated!"}2209gitlab # [ 349.275168] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:24.874Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}2210gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2211[]2212gitlab # [ 349.289712] nixbot[3255]: INFO: ::ffff:127.0.0.1:41606 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2213gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2214gitlab # [ 350.307679] nixbot[3255]: INFO: ::ffff:127.0.0.1:41612 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2215gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2216[]2217gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2218gitlab # [ 351.326607] nixbot[3255]: INFO: ::ffff:127.0.0.1:41620 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2219gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2220[]2221gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2222gitlab # [ 352.345178] nixbot[3255]: INFO: ::ffff:127.0.0.1:41632 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2223gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2224[]2225gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2226gitlab # [ 353.366583] nixbot[3255]: INFO: ::ffff:127.0.0.1:34924 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2227gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2228[]2229gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2230gitlab # [ 354.389317] nixbot[3255]: INFO: ::ffff:127.0.0.1:34938 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2231gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2232[]2233gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2234gitlab # [ 355.409808] nixbot[3255]: INFO: ::ffff:127.0.0.1:34954 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2235gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2236[]2237gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2238gitlab # [ 356.433372] nixbot[3255]: INFO: ::ffff:127.0.0.1:34968 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2239gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2240[]2241gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2242gitlab # [ 357.455232] nixbot[3255]: INFO: ::ffff:127.0.0.1:34974 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2243gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2244[]2245gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2246gitlab # [ 358.474976] nixbot[3255]: INFO: ::ffff:127.0.0.1:34978 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2247gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2248[]2249gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2250gitlab # [ 359.495772] nixbot[3255]: INFO: ::ffff:127.0.0.1:34980 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2251gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2252[]2253gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2254gitlab # [ 360.516872] nixbot[3255]: INFO: ::ffff:127.0.0.1:34990 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2255gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2256[]2257gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2258gitlab # [ 361.539405] nixbot[3255]: INFO: ::ffff:127.0.0.1:34994 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2259gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2260[]2261gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2262gitlab # [ 362.561500] nixbot[3255]: INFO: ::ffff:127.0.0.1:35002 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2263gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2264[]2265gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2266gitlab # [ 363.583614] nixbot[3255]: INFO: ::ffff:127.0.0.1:44590 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2267gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2268[]2269gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2270gitlab # [ 364.611391] nixbot[3255]: INFO: ::ffff:127.0.0.1:44606 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2271gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.03 seconds)2272[]2273gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2274gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2275[]2276gitlab # [ 365.632293] nixbot[3255]: INFO: ::ffff:127.0.0.1:44608 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2277gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2278gitlab # [ 366.648284] nixbot[3255]: INFO: ::ffff:127.0.0.1:44616 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2279gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2280[]2281gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2282gitlab # [ 367.667822] nixbot[3255]: INFO: ::ffff:127.0.0.1:44632 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2283gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2284[]2285gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2286gitlab # [ 368.686468] nixbot[3255]: INFO: ::ffff:127.0.0.1:44648 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2287gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2288[]2289gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2290gitlab # [ 369.705896] nixbot[3255]: INFO: ::ffff:127.0.0.1:44662 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2291gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2292[]2293gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2294gitlab # [ 370.727503] nixbot[3255]: INFO: ::ffff:127.0.0.1:44676 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2295gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2296[]2297gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2298gitlab # [ 371.747545] nixbot[3255]: INFO: ::ffff:127.0.0.1:44684 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2299gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2300[]2301gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2302gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2303[]2304gitlab # [ 372.770283] nixbot[3255]: INFO: ::ffff:127.0.0.1:44692 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2305gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2306gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2307[]2308gitlab # [ 373.787664] nixbot[3255]: INFO: ::ffff:127.0.0.1:37148 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2309gitlab # [ 374.181775] postgres[3788]: [3788] ERROR: relation "table_without_models" does not exist2310gitlab # [ 374.182290] postgres[3788]: [3788] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')2311gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2312gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2313[]2314gitlab # [ 374.811448] nixbot[3255]: INFO: ::ffff:127.0.0.1:37152 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2315gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2316gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2317[]2318gitlab # [ 375.832639] nixbot[3255]: INFO: ::ffff:127.0.0.1:37166 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2319gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2320gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2321[]2322gitlab # [ 376.856292] nixbot[3255]: INFO: ::ffff:127.0.0.1:37178 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2323gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2324gitlab # [ 377.882939] nixbot[3255]: INFO: ::ffff:127.0.0.1:37192 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2325gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.03 seconds)2326[]2327gitlab # [ 378.674338] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:54.274Z","message":"Booted Rails 7.2.3.1 application in production environment"}2328gitlab # [ 378.676906] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:54.276Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}2329gitlab # [ 378.679473] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:54.276Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}2330gitlab # [ 378.681715] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:54.276Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}2331gitlab # [ 378.684298] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:54.277Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}2332gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2333gitlab # [ 378.903769] nixbot[3255]: INFO: ::ffff:127.0.0.1:37198 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2334gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2335[]2336gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2337gitlab # [ 379.922643] nixbot[3255]: INFO: ::ffff:127.0.0.1:37212 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2338gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2339[]2340gitlab # [ 380.268640] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:55.868Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"0a23fba890b1da9babfd733c","created_at":"2026-09-01T10:36:40.251Z","trace_propagation_headers":{"sentry-trace":"5703dd92a86c4630b3ae32549a9dbc15-53bf5cbc22cc49cf","baggage":"sentry-trace_id=5703dd92a86c4630b3ae32549a9dbc15,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8NPW7TVGQWHDPJQSKCS7P","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:40.287Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-0a23fba890b1da9babfd733c: start","job_status":"start","queue_duration_s":75.581085,"scheduling_latency_s":75.581155}2341gitlab # [ 380.439187] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:56.039Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"0a23fba890b1da9babfd733c","created_at":"2026-09-01T10:36:40.251Z","trace_propagation_headers":{"sentry-trace":"5703dd92a86c4630b3ae32549a9dbc15-53bf5cbc22cc49cf","baggage":"sentry-trace_id=5703dd92a86c4630b3ae32549a9dbc15,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8NPW7TVGQWHDPJQSKCS7P","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:40.287Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-0a23fba890b1da9babfd733c: done: 0.170873 sec","job_status":"done","queue_duration_s":75.581085,"scheduling_latency_s":75.581155,"redis_calls":24,"redis_duration_s":0.006235,"redis_read_bytes":1038,"redis_write_bytes":2604,"redis_feature_flag_calls":14,"redis_feature_flag_duration_s":0.003466,"redis_feature_flag_read_bytes":1030,"redis_feature_flag_write_bytes":1439,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001532,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001237,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.075022384,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.170873,"completed_at":"2026-09-01T10:37:56.039Z","load_balancing_strategy":"primary","db_duration_s":0.038304,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2342gitlab # [ 380.440504] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:56.041Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"f578fe0cadb0a48700b0f70e","created_at":"2026-09-01T10:36:40.298Z","trace_propagation_headers":{"sentry-trace":"5703dd92a86c4630b3ae32549a9dbc15-53bf5cbc22cc49cf","baggage":"sentry-trace_id=5703dd92a86c4630b3ae32549a9dbc15,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8NPW7TVGQWHDPJQSKCS7P","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:40.306Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-f578fe0cadb0a48700b0f70e: start","job_status":"start","queue_duration_s":75.734711,"scheduling_latency_s":75.734734}2343gitlab # [ 380.536947] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:56.137Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"f578fe0cadb0a48700b0f70e","created_at":"2026-09-01T10:36:40.298Z","trace_propagation_headers":{"sentry-trace":"5703dd92a86c4630b3ae32549a9dbc15-53bf5cbc22cc49cf","baggage":"sentry-trace_id=5703dd92a86c4630b3ae32549a9dbc15,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8NPW7TVGQWHDPJQSKCS7P","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:40.306Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-f578fe0cadb0a48700b0f70e: done: 0.095987 sec","job_status":"done","queue_duration_s":75.734711,"scheduling_latency_s":75.734734,"redis_calls":14,"redis_duration_s":0.003551,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000829,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001683,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001039,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.069867,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.070808441,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.095987,"completed_at":"2026-09-01T10:37:56.136Z","load_balancing_strategy":"primary","db_duration_s":0.018185,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2344gitlab # [ 380.538800] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:56.139Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"7a699f83bd87434b93ae8bfe","created_at":"2026-09-01T10:36:40.998Z","trace_propagation_headers":{"sentry-trace":"5703dd92a86c4630b3ae32549a9dbc15-53bf5cbc22cc49cf","baggage":"sentry-trace_id=5703dd92a86c4630b3ae32549a9dbc15,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8NPW7TVGQWHDPJQSKCS7P","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:41.009Z","job_size_bytes":5,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-7a699f83bd87434b93ae8bfe: start","job_status":"start","queue_duration_s":75.129916,"scheduling_latency_s":75.129949}2345gitlab # [ 380.691148] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:56.291Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"7a699f83bd87434b93ae8bfe","created_at":"2026-09-01T10:36:40.998Z","trace_propagation_headers":{"sentry-trace":"5703dd92a86c4630b3ae32549a9dbc15-53bf5cbc22cc49cf","baggage":"sentry-trace_id=5703dd92a86c4630b3ae32549a9dbc15,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8NPW7TVGQWHDPJQSKCS7P","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:41.009Z","job_size_bytes":5,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-7a699f83bd87434b93ae8bfe: done: 0.152245 sec","job_status":"done","queue_duration_s":75.129916,"scheduling_latency_s":75.129949,"redis_calls":12,"redis_duration_s":0.003028,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000833,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001766,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000429,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.124787,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000339149,"duration_s":0.152245,"completed_at":"2026-09-01T10:37:56.291Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018016,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2346gitlab # [ 380.692406] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:56.292Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"7be661905d9f4fd0848adbac","created_at":"2026-09-01T10:36:41.051Z","trace_propagation_headers":{"sentry-trace":"5703dd92a86c4630b3ae32549a9dbc15-53bf5cbc22cc49cf","baggage":"sentry-trace_id=5703dd92a86c4630b3ae32549a9dbc15,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8NPW7TVGQWHDPJQSKCS7P","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:41.060Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-7be661905d9f4fd0848adbac: start","job_status":"start","queue_duration_s":75.232538,"scheduling_latency_s":75.232557}2347gitlab # [ 380.862226] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:56.462Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"7be661905d9f4fd0848adbac","created_at":"2026-09-01T10:36:41.051Z","trace_propagation_headers":{"sentry-trace":"5703dd92a86c4630b3ae32549a9dbc15-53bf5cbc22cc49cf","baggage":"sentry-trace_id=5703dd92a86c4630b3ae32549a9dbc15,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8NPW7TVGQWHDPJQSKCS7P","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:41.060Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-7be661905d9f4fd0848adbac: done: 0.169739 sec","job_status":"done","queue_duration_s":75.232538,"scheduling_latency_s":75.232557,"redis_calls":11,"redis_duration_s":0.002479,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00077,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001335,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000374,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.14,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.14,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.118227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000438324,"duration_s":0.169739,"completed_at":"2026-09-01T10:37:56.462Z","load_balancing_strategy":"primary","db_duration_s":0.052908,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2348gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2349gitlab # [ 380.863795] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:56.464Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"9bb5dfd9c1375ed5887f903e","created_at":"2026-09-01T10:36:41.064Z","trace_propagation_headers":{"sentry-trace":"5703dd92a86c4630b3ae32549a9dbc15-53bf5cbc22cc49cf","baggage":"sentry-trace_id=5703dd92a86c4630b3ae32549a9dbc15,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8NPW7TVGQWHDPJQSKCS7P","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:41.073Z","job_size_bytes":89,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-9bb5dfd9c1375ed5887f903e: start","job_status":"start","queue_duration_s":75.391053,"scheduling_latency_s":75.391072}2350gitlab # [ 380.939469] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:56.500Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"9bb5dfd9c1375ed5887f903e","created_at":"2026-09-01T10:36:41.064Z","trace_propagation_headers":{"sentry-trace":"5703dd92a86c4630b3ae32549a9dbc15-53bf5cbc22cc49cf","baggage":"sentry-trace_id=5703dd92a86c4630b3ae32549a9dbc15,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8NPW7TVGQWHDPJQSKCS7P","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:41.073Z","job_size_bytes":89,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-9bb5dfd9c1375ed5887f903e: done: 0.03647 sec","job_status":"done","queue_duration_s":75.391053,"scheduling_latency_s":75.391072,"redis_calls":11,"redis_duration_s":0.002724,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000733,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001535,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000456,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023343,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000489727,"duration_s":0.03647,"completed_at":"2026-09-01T10:37:56.500Z","load_balancing_strategy":"primary","db_duration_s":0.004908,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2351gitlab # [ 380.993943] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:56.502Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 7bdb1025a7f20513f1ec42f289b6f901a9fe07ea refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"82cdb1110ed3c01a441feb21","created_at":"2026-09-01T10:36:43.292Z","trace_propagation_headers":{"sentry-trace":"fcb371ab68e545d89550b8c507a29ce6-724d7222c3b94e47","baggage":"sentry-trace_id=fcb371ab68e545d89550b8c507a29ce6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8NT6V03XAPSJJAS0JE1RB","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:43.323Z","job_size_bytes":185,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-82cdb1110ed3c01a441feb21: start","job_status":"start","queue_duration_s":73.178844,"scheduling_latency_s":73.178868}2352gitlab # [ 380.994961] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:56.520Z","message":"changes: \"0000000000000000000000000000000000000000 7bdb1025a7f20513f1ec42f289b6f901a9fe07ea refs/heads/master\\n\""}2353gitlab # [ 380.996119] gitaly[932]: time="2026-09-01T10:37:56.557Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:38:06.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:56.551" grpc.time_ms=5.593 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2354gitlab # [ 380.997192] gitaly[932]: time="2026-09-01T10:37:56.587Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=620 command.oublock=0 command.real_time_ms=24 command.spawn_token_fork_ms=20 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-01T10:38:06.021" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:37:56.561" grpc.time_ms=25.43 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2355gitlab # [ 381.037055] gitaly[932]: time="2026-09-01T10:37:56.637Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-01T10:38:06.090" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:56.630" grpc.time_ms=6.71 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2356gitlab # [ 381.083565] nixbot[3255]: INFO: ::ffff:127.0.0.1:37218 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2357gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.16 seconds)2358[]2359gitlab # [ 381.094222] gitaly[932]: time="2026-09-01T10:37:56.694Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3933 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:37:56.694" grpc.time_ms=0.51 pid=932 span.kind=server system=grpc2360gitlab # [ 381.128987] gitaly[932]: time="2026-09-01T10:37:56.729Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3935 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:37:56.728" grpc.time_ms=0.574 pid=932 span.kind=server system=grpc2361gitlab # [ 381.146496] gitaly[932]: time="2026-09-01T10:37:56.747Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB diskcache=ab7a2fd3-54cc-45d7-be73-fcf575d274ad grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-01T10:38:06.132" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:37:56.673" pid=932 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root2362gitlab # [ 381.159719] gitaly[932]: time="2026-09-01T10:37:56.747Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=43 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4976 command.oublock=8 command.real_time_ms=69 command.spawn_token_fork_ms=0 command.system_time_ms=20 command.user_time_ms=23 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-01T10:38:06.132" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:37:56.673" grpc.time_ms=74.043 pid=932 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root2363gitlab # [ 381.166594] gitaly[932]: time="2026-09-01T10:37:56.767Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T10:38:06.226" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:37:56.766" grpc.time_ms=0.152 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2364gitlab # [ 381.203288] gitaly[932]: time="2026-09-01T10:37:56.803Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=933 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-01T10:38:06.256" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:56.797" grpc.time_ms=6.226 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2365gitlab # [ 381.276571] gitaly[932]: time="2026-09-01T10:37:56.877Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=339 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-01T10:38:26.072" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:56.873" grpc.time_ms=3.822 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2366gitlab # [ 381.300477] gitaly[932]: time="2026-09-01T10:37:56.901Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-01T10:38:06.010" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:37:56.900" grpc.time_ms=0.269 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2367gitlab # [ 381.306658] gitaly[932]: time="2026-09-01T10:37:56.907Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=342 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-01T10:38:06.012" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-01T10:37:56.903" grpc.time_ms=4.037 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2368gitlab # [ 381.347777] gitaly[932]: time="2026-09-01T10:37:56.948Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:38:06.053" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:56.943" grpc.time_ms=4.681 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2369gitlab # [ 381.375057] gitaly[932]: time="2026-09-01T10:37:56.975Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:38:06.080" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:56.970" grpc.time_ms=4.903 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2370gitlab # [ 381.396579] gitaly[932]: time="2026-09-01T10:37:56.997Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=960 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-01T10:38:26.187" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:56.988" grpc.time_ms=8.873 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2371gitlab # [ 381.866657] gitaly[932]: time="2026-09-01T10:37:57.467Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-01T10:38:07.005" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:57.465" grpc.time_ms=1.266 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2372gitlab # [ 381.910303] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:57.510Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 7bdb1025a7f20513f1ec42f289b6f901a9fe07ea refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"82cdb1110ed3c01a441feb21","created_at":"2026-09-01T10:36:43.292Z","trace_propagation_headers":{"sentry-trace":"fcb371ab68e545d89550b8c507a29ce6-724d7222c3b94e47","baggage":"sentry-trace_id=fcb371ab68e545d89550b8c507a29ce6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8NT6V03XAPSJJAS0JE1RB","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:43.323Z","job_size_bytes":185,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-82cdb1110ed3c01a441feb21: done: 1.007497 sec","job_status":"done","queue_duration_s":73.178844,"scheduling_latency_s":73.178868,"gitaly_calls":13,"gitaly_duration_s":0.288341,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.04508,"redis_read_bytes":2103,"redis_write_bytes":21751,"redis_cache_calls":15,"redis_cache_duration_s":0.003979,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.006652,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001574,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7839,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003666,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.020113,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.009096,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.041,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.213,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.233,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.041,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.635137,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000743111,"exclusive_lock_hold_duration_s":0.027215191,"duration_s":1.007497,"completed_at":"2026-09-01T10:37:57.510Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.067692,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2373gitlab # [ 381.918910] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:57.511Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","7bdb1025a7f20513f1ec42f289b6f901a9fe07ea e8d9360f6d71fb57cb350a711a26cf7895d79390 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"c6f82176509511d2490e63b5","created_at":"2026-09-01T10:36:50.785Z","trace_propagation_headers":{"sentry-trace":"76d23fcdcb984775a6db1bdb1776e475-81012cdfad8b46c7","baggage":"sentry-trace_id=76d23fcdcb984775a6db1bdb1776e475,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8P14TFMG1B28PPP39VJ7Z","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:50.790Z","job_size_bytes":185,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-c6f82176509511d2490e63b5: start","job_status":"start","queue_duration_s":66.721266,"scheduling_latency_s":66.721292}2374gitlab # [ 381.983375] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:57.524Z","message":"changes: \"7bdb1025a7f20513f1ec42f289b6f901a9fe07ea e8d9360f6d71fb57cb350a711a26cf7895d79390 refs/heads/master\\n\""}2375gitlab # [ 381.984396] gitaly[932]: time="2026-09-01T10:37:57.545Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:38:07.080" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:57.540" grpc.time_ms=4.817 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2376gitlab # [ 382.018282] gitaly[932]: time="2026-09-01T10:37:57.618Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=67 command.spawn_token_fork_ms=64 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-01T10:38:07.009" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:37:57.549" grpc.time_ms=68.543 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2377gitlab # [ 382.037237] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:57.622Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"da70c67e5de6e87a71e2d2c2","created_at":"2026-09-01T10:37:57.620Z","trace_propagation_headers":{"sentry-trace":"76d23fcdcb984775a6db1bdb1776e475-62be3ede22ba4390","baggage":"sentry-trace_id=76d23fcdcb984775a6db1bdb1776e475,sentry-release=Unknown"},"correlation_id":"01M1E8P14TFMG1B28PPP39VJ7Z","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"f2b275cdeaed0d51dbec7c50","job_size_bytes":46,"pid":3276,"job_status":"deduplicated","message":"ProjectCacheWorker JID-da70c67e5de6e87a71e2d2c2: deduplicated: until executing","deduplication.type":"until executing"}2378gitlab # [ 382.059172] gitaly[932]: time="2026-09-01T10:37:57.659Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=19 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-01T10:38:07.094" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:57.634" grpc.time_ms=24.797 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2379gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2380gitlab # [ 382.083689] gitaly[932]: time="2026-09-01T10:37:57.684Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=651 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-01T10:38:27.077" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:57.677" grpc.time_ms=6.655 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2381gitlab # [ 382.109745] nixbot[3255]: INFO: ::ffff:127.0.0.1:37232 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2382gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.02 seconds)2383[]2384gitlab # [ 382.114948] gitaly[932]: time="2026-09-01T10:37:57.715Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=346 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-01T10:38:07.171" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-01T10:37:57.711" grpc.time_ms=3.373 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2385gitlab # [ 382.127255] gitaly[932]: time="2026-09-01T10:37:57.727Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:38:07.182" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:57.722" grpc.time_ms=5.041 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2386gitlab # [ 382.201321] gitaly[932]: time="2026-09-01T10:37:57.801Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-01T10:38:07.260" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:57.800" grpc.time_ms=0.954 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2387gitlab # [ 382.222833] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:57.822Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","7bdb1025a7f20513f1ec42f289b6f901a9fe07ea e8d9360f6d71fb57cb350a711a26cf7895d79390 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"c6f82176509511d2490e63b5","created_at":"2026-09-01T10:36:50.785Z","trace_propagation_headers":{"sentry-trace":"76d23fcdcb984775a6db1bdb1776e475-81012cdfad8b46c7","baggage":"sentry-trace_id=76d23fcdcb984775a6db1bdb1776e475,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8P14TFMG1B28PPP39VJ7Z","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-01T10:36:50.790Z","job_size_bytes":185,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-c6f82176509511d2490e63b5: done: 0.310888 sec","job_status":"done","queue_duration_s":66.721266,"scheduling_latency_s":66.721292,"gitaly_calls":7,"gitaly_duration_s":0.162556,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.019476,"redis_read_bytes":500,"redis_write_bytes":11941,"redis_cache_calls":9,"redis_cache_duration_s":0.002338,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.000908,"redis_queues_read_bytes":8,"redis_queues_write_bytes":5718,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002659,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.006736,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.006835,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.016,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.016,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.122822,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000530793,"exclusive_lock_hold_duration_s":0.0137199,"duration_s":0.310888,"completed_at":"2026-09-01T10:37:57.822Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020148,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2388gitlab # [ 382.228923] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:57.825Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"7a74db26d8c57e51eceb1f48","created_at":"2026-09-01T10:36:52.122Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-e9d3dc33e2674887","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:52.130Z","job_size_bytes":65,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-7a74db26d8c57e51eceb1f48: start","job_status":"start","queue_duration_s":65.694213,"scheduling_latency_s":65.694246}2389gitlab # [ 382.304642] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:57.904Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"7a74db26d8c57e51eceb1f48","created_at":"2026-09-01T10:36:52.122Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-e9d3dc33e2674887","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:52.130Z","job_size_bytes":65,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-7a74db26d8c57e51eceb1f48: done: 0.079938 sec","job_status":"done","queue_duration_s":65.694213,"scheduling_latency_s":65.694246,"redis_calls":16,"redis_duration_s":0.0040490000000000005,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002053,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001542,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000454,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057698,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000361219,"duration_s":0.079938,"completed_at":"2026-09-01T10:37:57.904Z","load_balancing_strategy":"primary","db_duration_s":0.013921,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2390gitlab # [ 382.306793] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:57.906Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"d859b23e919ecf23453e38fd","created_at":"2026-09-01T10:36:52.133Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-e9d3dc33e2674887","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:52.146Z","job_size_bytes":65,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-d859b23e919ecf23453e38fd: start","job_status":"start","queue_duration_s":65.759627,"scheduling_latency_s":65.759652}2391gitlab # [ 382.403797] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:58.004Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"d859b23e919ecf23453e38fd","created_at":"2026-09-01T10:36:52.133Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-e9d3dc33e2674887","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:52.146Z","job_size_bytes":65,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-d859b23e919ecf23453e38fd: done: 0.097507 sec","job_status":"done","queue_duration_s":65.759627,"scheduling_latency_s":65.759652,"redis_calls":21,"redis_duration_s":0.005201,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.00249,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001364,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001347,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055406,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000340267,"duration_s":0.097507,"completed_at":"2026-09-01T10:37:58.003Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021031,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2392gitlab # [ 382.405431] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:58.005Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3a6282065f2851d1efbc8491","created_at":"2026-09-01T10:36:52.309Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-e9d3dc33e2674887","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:52.318Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-3a6282065f2851d1efbc8491: start","job_status":"start","queue_duration_s":65.686893,"scheduling_latency_s":65.68692}2393gitlab # [ 382.534055] gitaly[932]: time="2026-09-01T10:37:58.134Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8P2T0XK1F4TPTHNEXVHP1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:37:58.133" grpc.time_ms=0.584 pid=932 span.kind=server system=grpc2394gitlab # [ 382.565141] gitaly[932]: time="2026-09-01T10:37:58.165Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8P2T0XK1F4TPTHNEXVHP1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:37:58.165" grpc.time_ms=0.405 pid=932 span.kind=server system=grpc2395gitlab # [ 382.598339] gitaly[932]: time="2026-09-01T10:37:58.198Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8P2T0XK1F4TPTHNEXVHP1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:37:58.198" grpc.time_ms=0.36 pid=932 span.kind=server system=grpc2396gitlab # [ 382.618276] gitaly[932]: time="2026-09-01T10:37:58.218Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P2T0XK1F4TPTHNEXVHP1 diskcache=bdfd8431-2e31-4d42-9eeb-b0e10869c3c2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T10:38:28.015" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:58.115" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2397gitlab # [ 382.618780] gitaly[932]: time="2026-09-01T10:37:58.219Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=88 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7393 command.oublock=0 command.real_time_ms=99 command.spawn_token_fork_ms=0 command.system_time_ms=54 command.user_time_ms=34 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P2T0XK1F4TPTHNEXVHP1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T10:38:28.015" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:58.115" grpc.time_ms=103.044 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2398gitlab # [ 382.713952] gitaly[932]: time="2026-09-01T10:37:58.314Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8P2T0XK1F4TPTHNEXVHP1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-01T10:38:28.213" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:58.313" grpc.time_ms=0.897 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2399gitlab # [ 382.743712] gitaly[932]: time="2026-09-01T10:37:58.344Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P2T0XK1F4TPTHNEXVHP1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:28.243" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:58.343" grpc.time_ms=0.541 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2400gitlab # [ 382.753233] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:58.353Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3a6282065f2851d1efbc8491","created_at":"2026-09-01T10:36:52.309Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-e9d3dc33e2674887","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:52.318Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3276,"job_status":"rescheduled","message":"PipelineProcessWorker JID-3a6282065f2851d1efbc8491: rescheduled"}2401gitlab # [ 382.761758] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:58.361Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3a6282065f2851d1efbc8491","created_at":"2026-09-01T10:36:52.309Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-e9d3dc33e2674887","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T10:36:52.318Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-3a6282065f2851d1efbc8491: done: 0.356425 sec","job_status":"done","queue_duration_s":65.686893,"scheduling_latency_s":65.68692,"gitaly_calls":3,"gitaly_duration_s":0.111502,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.024002000000000002,"redis_read_bytes":462,"redis_write_bytes":29293,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001168,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001144,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.008722,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.003602,"redis_queues_read_bytes":39,"redis_queues_write_bytes":15581,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.005107,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000472,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":165,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000568,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002051,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.015,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.018,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.019,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.185429,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.30604169000000003,"duration_s":0.356425,"completed_at":"2026-09-01T10:37:58.361Z","load_balancing_strategy":"primary","db_duration_s":0.047055,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2402gitlab # [ 382.838988] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:58.363Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"f2b275cdeaed0d51dbec7c50","created_at":"2026-09-01T10:37:56.590Z","trace_propagation_headers":{"sentry-trace":"fcb371ab68e545d89550b8c507a29ce6-e53c1cf24a2143a0","baggage":"sentry-trace_id=fcb371ab68e545d89550b8c507a29ce6,sentry-release=Unknown"},"correlation_id":"01M1E8NT6V03XAPSJJAS0JE1RB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:56.603Z","job_size_bytes":46,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-f2b275cdeaed0d51dbec7c50: start","job_status":"start","queue_duration_s":1.760422,"scheduling_latency_s":1.760447}2403gitlab # [ 382.839931] gitaly[932]: time="2026-09-01T10:37:58.406Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-01T16:37:58.403" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:37:58.404" grpc.time_ms=1.749 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2404gitlab # [ 382.840828] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:58.439Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"f2b275cdeaed0d51dbec7c50","created_at":"2026-09-01T10:37:56.590Z","trace_propagation_headers":{"sentry-trace":"fcb371ab68e545d89550b8c507a29ce6-e53c1cf24a2143a0","baggage":"sentry-trace_id=fcb371ab68e545d89550b8c507a29ce6,sentry-release=Unknown"},"correlation_id":"01M1E8NT6V03XAPSJJAS0JE1RB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:56.603Z","job_size_bytes":46,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-f2b275cdeaed0d51dbec7c50: done: 0.076386 sec","job_status":"done","queue_duration_s":1.760422,"scheduling_latency_s":1.760447,"gitaly_calls":2,"gitaly_duration_s":0.011028,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.006658,"redis_read_bytes":69,"redis_write_bytes":4888,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001186,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.00052,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001501,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.002527,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000924,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047358,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000355911,"duration_s":0.076386,"completed_at":"2026-09-01T10:37:58.439Z","load_balancing_strategy":"primary","db_duration_s":0.00807,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2405gitlab # [ 382.917535] gitaly[932]: time="2026-09-01T10:37:58.418Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-01T10:38:28.312" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:58.413" grpc.time_ms=5.454 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2406gitlab # [ 382.918942] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:58.441Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"b4d5717f4f9c73d5f461f513","created_at":"2026-09-01T10:37:56.619Z","trace_propagation_headers":{"sentry-trace":"fcb371ab68e545d89550b8c507a29ce6-e53c1cf24a2143a0","baggage":"sentry-trace_id=fcb371ab68e545d89550b8c507a29ce6,sentry-release=Unknown"},"correlation_id":"01M1E8NT6V03XAPSJJAS0JE1RB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:56.625Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-b4d5717f4f9c73d5f461f513: start","job_status":"start","queue_duration_s":1.816034,"scheduling_latency_s":1.816059}2407gitlab # [ 382.989621] gitaly[932]: time="2026-09-01T10:37:58.590Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB diskcache=db8d42b3-9373-4820-96b9-1f14760f0bdf grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-01T16:37:58.465" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:58.465" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2408gitlab # [ 382.990239] gitaly[932]: time="2026-09-01T10:37:58.590Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1278 command.oublock=0 command.real_time_ms=233 command.spawn_token_fork_ms=109 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-01T16:37:58.465" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:58.465" grpc.time_ms=124.496 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2409gitlab # [ 383.021306] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:58.621Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"b4d5717f4f9c73d5f461f513","created_at":"2026-09-01T10:37:56.619Z","trace_propagation_headers":{"sentry-trace":"fcb371ab68e545d89550b8c507a29ce6-e53c1cf24a2143a0","baggage":"sentry-trace_id=fcb371ab68e545d89550b8c507a29ce6,sentry-release=Unknown"},"correlation_id":"01M1E8NT6V03XAPSJJAS0JE1RB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:56.625Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-b4d5717f4f9c73d5f461f513: done: 0.179805 sec","job_status":"done","queue_duration_s":1.816034,"scheduling_latency_s":1.816059,"gitaly_calls":1,"gitaly_duration_s":0.12748,"redis_calls":15,"redis_duration_s":0.003303,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000479,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001219,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000579,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001026,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.009,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.016,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035219,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.156646167,"duration_s":0.179805,"completed_at":"2026-09-01T10:37:58.621Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011304,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2410gitlab # [ 383.024803] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:58.623Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"9f7fbd6810d594cc6d93b745","created_at":"2026-09-01T10:37:56.762Z","trace_propagation_headers":{"sentry-trace":"fcb371ab68e545d89550b8c507a29ce6-e53c1cf24a2143a0","baggage":"sentry-trace_id=fcb371ab68e545d89550b8c507a29ce6,sentry-release=Unknown"},"correlation_id":"01M1E8NT6V03XAPSJJAS0JE1RB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:56.764Z","job_size_bytes":23,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-9f7fbd6810d594cc6d93b745: start","job_status":"start","queue_duration_s":1.859067,"scheduling_latency_s":1.85909}2411gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2412gitlab # [ 383.080431] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:58.667Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"9f7fbd6810d594cc6d93b745","created_at":"2026-09-01T10:37:56.762Z","trace_propagation_headers":{"sentry-trace":"fcb371ab68e545d89550b8c507a29ce6-e53c1cf24a2143a0","baggage":"sentry-trace_id=fcb371ab68e545d89550b8c507a29ce6,sentry-release=Unknown"},"correlation_id":"01M1E8NT6V03XAPSJJAS0JE1RB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:56.764Z","job_size_bytes":23,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-9f7fbd6810d594cc6d93b745: done: 0.043906 sec","job_status":"done","queue_duration_s":1.859067,"scheduling_latency_s":1.85909,"gitaly_calls":1,"gitaly_duration_s":0.007703,"redis_calls":16,"redis_duration_s":0.004095,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000316,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001091,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001878,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00081,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028481,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043906,"completed_at":"2026-09-01T10:37:58.667Z","load_balancing_strategy":"primary","db_duration_s":0.002973,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2413gitlab # [ 383.101167] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:58.668Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"11d291a9453ba0fc5ceb252c","created_at":"2026-09-01T10:37:57.255Z","trace_propagation_headers":{"sentry-trace":"fcb371ab68e545d89550b8c507a29ce6-e53c1cf24a2143a0","baggage":"sentry-trace_id=fcb371ab68e545d89550b8c507a29ce6,sentry-release=Unknown"},"correlation_id":"01M1E8NT6V03XAPSJJAS0JE1RB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2d0da9b17538ebc263b77cb728dd5e2200a0f681e9d01e111ec59f79b48852be","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:57.260Z","job_size_bytes":467,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-11d291a9453ba0fc5ceb252c: start","job_status":"start","queue_duration_s":1.408374,"scheduling_latency_s":1.408397}2414gitlab # [ 383.157909] gitaly[932]: time="2026-09-01T10:37:58.659Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=653 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-01T10:38:28.553" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:58.653" grpc.time_ms=5.342 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2415gitlab # [ 383.158949] gitaly[932]: time="2026-09-01T10:37:58.693Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:28.592" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:58.692" grpc.time_ms=0.644 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2416gitlab # [ 383.159888] gitaly[932]: time="2026-09-01T10:37:58.700Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-01T10:38:08.235" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:58.695" grpc.time_ms=4.503 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2417gitlab # [ 383.230197] nixbot[3255]: INFO: ::ffff:127.0.0.1:40678 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2418gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.12 seconds)2419[]2420gitlab # [ 383.248959] gitaly[932]: time="2026-09-01T10:37:58.849Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-01T10:38:28.748" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:58.848" grpc.time_ms=0.874 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2421gitlab # [ 383.255621] gitaly[932]: time="2026-09-01T10:37:58.856Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=92 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-01T10:38:28.753" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:58.853" grpc.time_ms=2.083 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2422gitlab # [ 383.334561] gitaly[932]: time="2026-09-01T10:37:58.935Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:38:08.039" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:58.930" grpc.time_ms=4.955 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2423gitlab # [ 383.955661] gitaly[932]: time="2026-09-01T10:37:59.556Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=959 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-01T10:38:29.448" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:59.548" grpc.time_ms=7.682 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2424gitlab # [ 383.971466] gitaly[932]: time="2026-09-01T10:37:59.572Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8NT6V03XAPSJJAS0JE1RB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-01T10:38:29.461" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:59.562" grpc.time_ms=9.682 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2425gitlab # [ 384.024504] sidekiq-cluster[3276]: {"severity":"WARN","time":"2026-09-01T10:37:59.624Z","class":"CreatePipelineWorker","correlation_id":"01M1E8NT6V03XAPSJJAS0JE1RB","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"7bdb1025a7f20513f1ec42f289b6f901a9fe07ea","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"7bdb1025a7f20513f1ec42f289b6f901a9fe07ea","ref":"refs/heads/master"},"retry":0}2426gitlab # [ 384.032829] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:59.633Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"11d291a9453ba0fc5ceb252c","created_at":"2026-09-01T10:37:57.255Z","trace_propagation_headers":{"sentry-trace":"fcb371ab68e545d89550b8c507a29ce6-e53c1cf24a2143a0","baggage":"sentry-trace_id=fcb371ab68e545d89550b8c507a29ce6,sentry-release=Unknown"},"correlation_id":"01M1E8NT6V03XAPSJJAS0JE1RB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2d0da9b17538ebc263b77cb728dd5e2200a0f681e9d01e111ec59f79b48852be","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:57.260Z","job_size_bytes":467,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-11d291a9453ba0fc5ceb252c: done: 0.964344 sec","job_status":"done","queue_duration_s":1.408374,"scheduling_latency_s":1.408397,"gitaly_calls":7,"gitaly_duration_s":0.07417,"redis_calls":51,"redis_duration_s":0.010579,"redis_read_bytes":169,"redis_write_bytes":4167,"redis_cache_calls":4,"redis_cache_duration_s":0.000915,"redis_cache_read_bytes":10,"redis_cache_write_bytes":342,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.004264,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001562,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002847,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000991,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.069,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.064,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.778682,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000386362,"duration_s":0.964344,"completed_at":"2026-09-01T10:37:59.632Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.084104,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2427gitlab # [ 384.043284] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:59.634Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"fa9f290b-a9f5-4ac4-95a6-f0b12590a122\"}"],"class":"WebHookWorker","jid":"03b53269eb86b5efa8774a37","created_at":"2026-09-01T10:37:57.440Z","trace_propagation_headers":{"sentry-trace":"fcb371ab68e545d89550b8c507a29ce6-e53c1cf24a2143a0","baggage":"sentry-trace_id=fcb371ab68e545d89550b8c507a29ce6,sentry-release=Unknown"},"correlation_id":"01M1E8NT6V03XAPSJJAS0JE1RB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:218442b974a410529908108eaef6fb5c9558f11e78c5988a75c3583dc392bb48","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:57.445Z","job_size_bytes":1751,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-03b53269eb86b5efa8774a37: start","job_status":"start","queue_duration_s":2.188977,"scheduling_latency_s":2.189004}2428gitlab # [ 384.142805] nixbot[3255]: INFO: ::ffff:127.0.0.1:40694 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2429gitlab # [ 384.159946] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:59.759Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"fa9f290b-a9f5-4ac4-95a6-f0b12590a122\"}"],"class":"WebHookWorker","jid":"03b53269eb86b5efa8774a37","created_at":"2026-09-01T10:37:57.440Z","trace_propagation_headers":{"sentry-trace":"fcb371ab68e545d89550b8c507a29ce6-e53c1cf24a2143a0","baggage":"sentry-trace_id=fcb371ab68e545d89550b8c507a29ce6,sentry-release=Unknown"},"correlation_id":"01M1E8NT6V03XAPSJJAS0JE1RB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:218442b974a410529908108eaef6fb5c9558f11e78c5988a75c3583dc392bb48","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:57.445Z","job_size_bytes":1751,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-03b53269eb86b5efa8774a37: done: 0.125233 sec","job_status":"done","queue_duration_s":2.188977,"scheduling_latency_s":2.189004,"redis_calls":19,"redis_duration_s":0.004256,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000898,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000334,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001928,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001096,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.03907284299998537,"cpu_s":0.070673,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000592813,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.125233,"completed_at":"2026-09-01T10:37:59.759Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003469,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2430gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2431gitlab # [ 384.229806] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:59.829Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"1a4ef5a9f6aafa2d9ba27c35","created_at":"2026-09-01T10:37:57.627Z","trace_propagation_headers":{"sentry-trace":"76d23fcdcb984775a6db1bdb1776e475-62be3ede22ba4390","baggage":"sentry-trace_id=76d23fcdcb984775a6db1bdb1776e475,sentry-release=Unknown"},"correlation_id":"01M1E8P14TFMG1B28PPP39VJ7Z","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"b4d5717f4f9c73d5f461f513","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:57.629Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-1a4ef5a9f6aafa2d9ba27c35: start","job_status":"start","queue_duration_s":2.200025,"scheduling_latency_s":2.200057}2432gitlab # [ 384.269610] nixbot[3255]: INFO: ::ffff:127.0.0.1:40702 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2433gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.04 seconds)2434[]2435gitlab # [ 384.283155] gitaly[932]: time="2026-09-01T10:37:59.882Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z diskcache=909cb9f6-ec53-497f-a67d-0f7a8dbb1a30 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-01T16:37:59.876" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:59.876" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2436gitlab # [ 384.283658] gitaly[932]: time="2026-09-01T10:37:59.883Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=316 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-01T16:37:59.876" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:59.876" grpc.time_ms=6.481 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2437gitlab # [ 384.302487] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:59.902Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"1a4ef5a9f6aafa2d9ba27c35","created_at":"2026-09-01T10:37:57.627Z","trace_propagation_headers":{"sentry-trace":"76d23fcdcb984775a6db1bdb1776e475-62be3ede22ba4390","baggage":"sentry-trace_id=76d23fcdcb984775a6db1bdb1776e475,sentry-release=Unknown"},"correlation_id":"01M1E8P14TFMG1B28PPP39VJ7Z","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"b4d5717f4f9c73d5f461f513","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:57.629Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-1a4ef5a9f6aafa2d9ba27c35: done: 0.072664 sec","job_status":"done","queue_duration_s":2.200025,"scheduling_latency_s":2.200057,"gitaly_calls":1,"gitaly_duration_s":0.010697,"redis_calls":12,"redis_duration_s":0.006085999999999999,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001487,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.003283,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001316,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024518,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.02876008,"duration_s":0.072664,"completed_at":"2026-09-01T10:37:59.902Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006518,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2438gitlab # [ 384.317926] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:37:59.905Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"852aa3d264d91c098624a261","created_at":"2026-09-01T10:37:57.767Z","trace_propagation_headers":{"sentry-trace":"76d23fcdcb984775a6db1bdb1776e475-62be3ede22ba4390","baggage":"sentry-trace_id=76d23fcdcb984775a6db1bdb1776e475,sentry-release=Unknown"},"correlation_id":"01M1E8P14TFMG1B28PPP39VJ7Z","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c6ae6a7a64f9c812952cca9c5c9d65a54a79206c452bf34952ce97ad2bf917df","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:57.769Z","job_size_bytes":467,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-852aa3d264d91c098624a261: start","job_status":"start","queue_duration_s":2.135763,"scheduling_latency_s":2.135794}2439gitlab # [ 384.380426] gitaly[932]: time="2026-09-01T10:37:59.931Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:29.830" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:59.930" grpc.time_ms=0.582 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2440gitlab # [ 384.381775] gitaly[932]: time="2026-09-01T10:37:59.938Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-01T10:38:09.043" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:59.933" grpc.time_ms=4.573 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2441gitlab # [ 384.382819] gitaly[932]: time="2026-09-01T10:37:59.969Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-01T10:38:29.868" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:59.968" grpc.time_ms=0.7 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2442gitlab # [ 384.394318] gitaly[932]: time="2026-09-01T10:37:59.975Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=326 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-01T10:38:29.872" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:37:59.972" grpc.time_ms=2.993 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2443gitlab # [ 384.465296] gitaly[932]: time="2026-09-01T10:38:00.002Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=5 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:38:09.105" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:37:59.996" grpc.time_ms=6.322 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2444gitlab # [ 384.625833] gitaly[932]: time="2026-09-01T10:38:00.226Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8R5JMHS0ZFJKHRVCS5YRJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:38:00.220" grpc.time_ms=5.072 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2445gitlab # [ 384.627542] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:00 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 4632446gitlab # [ 384.857470] gitaly[932]: time="2026-09-01T10:38:00.458Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=957 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-01T10:38:30.348" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:00.448" grpc.time_ms=9.008 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2447gitlab # [ 384.870684] gitaly[932]: time="2026-09-01T10:38:00.471Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8P14TFMG1B28PPP39VJ7Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-01T10:38:30.361" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:00.462" grpc.time_ms=8.892 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2448gitlab # [ 384.894334] sidekiq-cluster[3276]: {"severity":"WARN","time":"2026-09-01T10:38:00.494Z","class":"CreatePipelineWorker","correlation_id":"01M1E8P14TFMG1B28PPP39VJ7Z","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"7bdb1025a7f20513f1ec42f289b6f901a9fe07ea","after":"e8d9360f6d71fb57cb350a711a26cf7895d79390","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"e8d9360f6d71fb57cb350a711a26cf7895d79390","ref":"refs/heads/master"},"retry":0}2449gitlab # [ 384.899590] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:00.499Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"852aa3d264d91c098624a261","created_at":"2026-09-01T10:37:57.767Z","trace_propagation_headers":{"sentry-trace":"76d23fcdcb984775a6db1bdb1776e475-62be3ede22ba4390","baggage":"sentry-trace_id=76d23fcdcb984775a6db1bdb1776e475,sentry-release=Unknown"},"correlation_id":"01M1E8P14TFMG1B28PPP39VJ7Z","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c6ae6a7a64f9c812952cca9c5c9d65a54a79206c452bf34952ce97ad2bf917df","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:57.769Z","job_size_bytes":467,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-852aa3d264d91c098624a261: done: 0.594025 sec","job_status":"done","queue_duration_s":2.135763,"scheduling_latency_s":2.135794,"gitaly_calls":7,"gitaly_duration_s":0.064599,"redis_calls":33,"redis_duration_s":0.00554,"redis_read_bytes":185,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000619,"redis_cache_read_bytes":44,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001267,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002614,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00104,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.497782,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.594025,"completed_at":"2026-09-01T10:38:00.499Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016745,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2450gitlab # [ 384.953222] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:00.504Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"69deb546-9759-4a90-aa8b-ad652a01d32a\"}"],"class":"WebHookWorker","jid":"b45380ab5ddcae85a44e55df","created_at":"2026-09-01T10:37:57.786Z","trace_propagation_headers":{"sentry-trace":"76d23fcdcb984775a6db1bdb1776e475-62be3ede22ba4390","baggage":"sentry-trace_id=76d23fcdcb984775a6db1bdb1776e475,sentry-release=Unknown"},"correlation_id":"01M1E8P14TFMG1B28PPP39VJ7Z","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:033d3941012e3d5c3bd9eefbe663431972d020597960d903f38e84e56f885513","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:57.788Z","job_size_bytes":1749,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-b45380ab5ddcae85a44e55df: start","job_status":"start","queue_duration_s":2.715749,"scheduling_latency_s":2.715787}2451gitlab # [ 384.982416] gitaly[932]: time="2026-09-01T10:38:00.582Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R61QGQHW4JGHK37KACDY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:38:00.507" pid=932 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2452gitlab # [ 384.983312] gitaly[932]: time="2026-09-01T10:38:00.583Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=73 command.spawn_token_fork_ms=70 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R61QGQHW4JGHK37KACDY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:38:00.507" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2453gitlab # [ 384.985740] gitaly[932]: time="2026-09-01T10:38:00.583Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=73 command.spawn_token_fork_ms=70 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R61QGQHW4JGHK37KACDY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:38:00.507" grpc.time_ms=75.71 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2454gitlab # [ 384.986627] gitlab-workhorse[832]: time="2026-09-01T10:38:00Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672455gitlab # [ 384.987462] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:00 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 3372456gitlab # [ 385.018180] nixbot[3255]: INFO: ::ffff:127.0.0.1:40704 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2457gitlab # [ 385.030263] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:00.629Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"69deb546-9759-4a90-aa8b-ad652a01d32a\"}"],"class":"WebHookWorker","jid":"b45380ab5ddcae85a44e55df","created_at":"2026-09-01T10:37:57.786Z","trace_propagation_headers":{"sentry-trace":"76d23fcdcb984775a6db1bdb1776e475-62be3ede22ba4390","baggage":"sentry-trace_id=76d23fcdcb984775a6db1bdb1776e475,sentry-release=Unknown"},"correlation_id":"01M1E8P14TFMG1B28PPP39VJ7Z","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:033d3941012e3d5c3bd9eefbe663431972d020597960d903f38e84e56f885513","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:57.788Z","job_size_bytes":1749,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-b45380ab5ddcae85a44e55df: done: 0.124839 sec","job_status":"done","queue_duration_s":2.715749,"scheduling_latency_s":2.715787,"redis_calls":14,"redis_duration_s":0.002956,"redis_read_bytes":26,"redis_write_bytes":4684,"redis_queues_calls":2,"redis_queues_duration_s":0.00032,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001633,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001003,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.009268496000004234,"cpu_s":0.02484,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.124839,"completed_at":"2026-09-01T10:38:00.629Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002242,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2458gitlab # [ 385.041337] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:00.635Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c0529da268ab75ab4d49b917","created_at":"2026-09-01T10:37:58.099Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.103Z","job_size_bytes":23,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c0529da268ab75ab4d49b917: start","job_status":"start","queue_duration_s":2.531574,"scheduling_latency_s":2.531604}2459gitlab # [ 385.126478] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:00.726Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c0529da268ab75ab4d49b917","created_at":"2026-09-01T10:37:58.099Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.103Z","job_size_bytes":23,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c0529da268ab75ab4d49b917: done: 0.091476 sec","job_status":"done","queue_duration_s":2.531574,"scheduling_latency_s":2.531604,"redis_calls":10,"redis_duration_s":0.005986,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000935,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003114,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001937,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.06011,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000421282,"duration_s":0.091476,"completed_at":"2026-09-01T10:38:00.726Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016712,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2460gitlab # [ 385.157485] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:00.730Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d4ed6e0d759c1c6917926710","created_at":"2026-09-01T10:37:58.224Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.229Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d4ed6e0d759c1c6917926710: start","job_status":"start","queue_duration_s":2.500931,"scheduling_latency_s":2.500972}2461gitlab # [ 385.168228] nixbot[3255]: {"ts": "2026-09-01T10:38:00.768Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}2462gitlab # [ 385.204666] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:00.804Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d4ed6e0d759c1c6917926710","created_at":"2026-09-01T10:37:58.224Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.229Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d4ed6e0d759c1c6917926710: done: 0.074047 sec","job_status":"done","queue_duration_s":2.500931,"scheduling_latency_s":2.500972,"redis_calls":10,"redis_duration_s":0.006341,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000924,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003329,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002088,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028571,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000512355,"duration_s":0.074047,"completed_at":"2026-09-01T10:38:00.804Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008343,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2463gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2464gitlab # [ 385.212806] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:00.806Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"27a2285759791fb93c821bde","created_at":"2026-09-01T10:37:58.239Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.246Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-27a2285759791fb93c821bde: start","job_status":"start","queue_duration_s":2.560749,"scheduling_latency_s":2.560779}2465gitlab # [ 385.270847] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:00.848Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"27a2285759791fb93c821bde","created_at":"2026-09-01T10:37:58.239Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.246Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-27a2285759791fb93c821bde: done: 0.041525 sec","job_status":"done","queue_duration_s":2.560749,"scheduling_latency_s":2.560779,"redis_calls":10,"redis_duration_s":0.002837,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000615,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00171,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000512,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027336,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000407035,"duration_s":0.041525,"completed_at":"2026-09-01T10:38:00.848Z","load_balancing_strategy":"primary","db_duration_s":0.006244,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2466gitlab # [ 385.281236] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:00.850Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"fb929b9d5b9369e4ebb4d88d","created_at":"2026-09-01T10:37:58.257Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.260Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-fb929b9d5b9369e4ebb4d88d: start","job_status":"start","queue_duration_s":2.589322,"scheduling_latency_s":2.589351}2467gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.09 seconds)2468[]2469gitlab # [ 385.355183] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:00.907Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"fb929b9d5b9369e4ebb4d88d","created_at":"2026-09-01T10:37:58.257Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.260Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-fb929b9d5b9369e4ebb4d88d: done: 0.056733 sec","job_status":"done","queue_duration_s":2.589322,"scheduling_latency_s":2.589351,"redis_calls":13,"redis_duration_s":0.0070209999999999995,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000841,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002812,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002584,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000784,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029748,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000421282,"duration_s":0.056733,"completed_at":"2026-09-01T10:38:00.906Z","load_balancing_strategy":"primary","db_duration_s":0.014323,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2470gitlab # [ 385.363286] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:00.909Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"b7e2228268ee2af0d25305af","created_at":"2026-09-01T10:37:58.261Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.264Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-b7e2228268ee2af0d25305af: start","job_status":"start","queue_duration_s":2.644635,"scheduling_latency_s":2.644658}2471gitlab # [ 385.432230] nixbot[3255]: INFO: ::ffff:127.0.0.1:40706 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2472gitlab # [ 385.433642] gitaly[932]: time="2026-09-01T10:38:01.018Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R6JPHX6ZNE72J76NGB1X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:31.017" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:01.017" grpc.time_ms=1.064 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2473gitlab # [ 385.434953] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:00.946Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"b7e2228268ee2af0d25305af","created_at":"2026-09-01T10:37:58.261Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.264Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-b7e2228268ee2af0d25305af: done: 0.037391 sec","job_status":"done","queue_duration_s":2.644635,"scheduling_latency_s":2.644658,"redis_calls":10,"redis_duration_s":0.004059999999999999,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00046,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003109,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000491,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020516,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001257701,"duration_s":0.037391,"completed_at":"2026-09-01T10:38:00.946Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00807,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2474gitlab # [ 385.509229] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:00.948Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"28330d1253f3854013dc6dc4","created_at":"2026-09-01T10:37:58.264Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.268Z","job_size_bytes":19,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-28330d1253f3854013dc6dc4: start","job_status":"start","queue_duration_s":2.680258,"scheduling_latency_s":2.68035}2475gitlab # [ 385.509712] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.002Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"28330d1253f3854013dc6dc4","created_at":"2026-09-01T10:37:58.264Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.268Z","job_size_bytes":19,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-28330d1253f3854013dc6dc4: done: 0.053654 sec","job_status":"done","queue_duration_s":2.680258,"scheduling_latency_s":2.68035,"redis_calls":11,"redis_duration_s":0.010421,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001279,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.007711,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001431,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000799822,"duration_s":0.053654,"completed_at":"2026-09-01T10:38:01.002Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003857,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2476gitlab # [ 385.579238] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.004Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"9861fdba17c17d6e800fd09a","created_at":"2026-09-01T10:37:58.270Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.274Z","job_size_bytes":66,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-9861fdba17c17d6e800fd09a: start","job_status":"start","queue_duration_s":2.729919,"scheduling_latency_s":2.729947}2477gitlab # [ 385.580663] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.039Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"9861fdba17c17d6e800fd09a","created_at":"2026-09-01T10:37:58.270Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.274Z","job_size_bytes":66,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-9861fdba17c17d6e800fd09a: done: 0.035367 sec","job_status":"done","queue_duration_s":2.729919,"scheduling_latency_s":2.729947,"redis_calls":10,"redis_duration_s":0.005203,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000444,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003311,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001448,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019965,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002343035,"duration_s":0.035367,"completed_at":"2026-09-01T10:38:01.039Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004616,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2478gitlab # [ 385.671869] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.042Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"e6633f40d27152c3b06a315e","created_at":"2026-09-01T10:37:58.274Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.278Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-e6633f40d27152c3b06a315e: start","job_status":"start","queue_duration_s":2.763814,"scheduling_latency_s":2.763835}2479gitlab # [ 385.678216] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:01 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 3552480gitlab # [ 385.679308] gitlab-workhorse[832]: time="2026-09-01T10:38:01Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672481gitlab # [ 385.680300] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:01 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 572482gitlab # [ 385.680937] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:01 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 502483gitlab # [ 385.681859] gitaly[932]: time="2026-09-01T10:38:01.115Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=157 command.spawn_token_fork_ms=156 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8R6HRZ3FECC6DAAT2CC60 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:38:00.956" grpc.time_ms=159.055 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2484gitlab # [ 385.747092] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.090Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"e6633f40d27152c3b06a315e","created_at":"2026-09-01T10:37:58.274Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.278Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-e6633f40d27152c3b06a315e: done: 0.048294 sec","job_status":"done","queue_duration_s":2.763814,"scheduling_latency_s":2.763835,"redis_calls":10,"redis_duration_s":0.003382,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000472,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001258,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001652,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000384127,"duration_s":0.048294,"completed_at":"2026-09-01T10:38:01.090Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006074,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2485gitlab # [ 385.750395] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.091Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"cb21e7f87e115c02f90c41d3","created_at":"2026-09-01T10:37:58.284Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.288Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-cb21e7f87e115c02f90c41d3: start","job_status":"start","queue_duration_s":2.803583,"scheduling_latency_s":2.803607}2486gitlab # [ 385.826194] gitlab-workhorse[832]: time="2026-09-01T10:38:01Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2162487gitlab # [ 385.827834] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:01 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 692488gitlab # [ 385.829277] gitaly[932]: time="2026-09-01T10:38:01.175Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R6WY5R1JRTDCV7YCKE7J grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:38:01.167" pid=932 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2489gitlab # [ 385.830315] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.131Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"cb21e7f87e115c02f90c41d3","created_at":"2026-09-01T10:37:58.284Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.288Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-cb21e7f87e115c02f90c41d3: done: 0.039247 sec","job_status":"done","queue_duration_s":2.803583,"scheduling_latency_s":2.803607,"redis_calls":10,"redis_duration_s":0.002168,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00036,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001351,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000457,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019936,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000337193,"duration_s":0.039247,"completed_at":"2026-09-01T10:38:01.131Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013691,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2490gitlab # [ 385.884794] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.132Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"4c74fede3401224658bffb91","created_at":"2026-09-01T10:37:58.288Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.292Z","job_size_bytes":6,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-4c74fede3401224658bffb91: start","job_status":"start","queue_duration_s":2.84007,"scheduling_latency_s":2.840092}2491gitlab # [ 385.885795] gitaly[932]: time="2026-09-01T10:38:01.175Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R6WY5R1JRTDCV7YCKE7J grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:38:01.167" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2492gitlab # [ 385.886599] nixbot[3255]: {"ts": "2026-09-01T10:38:01.436Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/7bdb1025a7f20513f1ec42f289b6f901a9fe07ea \"HTTP/1.1 201 Created\""}2493gitlab # [ 385.887637] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.157Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"4c74fede3401224658bffb91","created_at":"2026-09-01T10:37:58.288Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.292Z","job_size_bytes":6,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-4c74fede3401224658bffb91: done: 0.024352 sec","job_status":"done","queue_duration_s":2.84007,"scheduling_latency_s":2.840092,"redis_calls":10,"redis_duration_s":0.002083,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00036,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001345,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000378,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01444,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000426032,"duration_s":0.024352,"completed_at":"2026-09-01T10:38:01.157Z","load_balancing_strategy":"primary","db_duration_s":0.001749,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2494gitlab # [ 385.950129] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.159Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"371b125806cba7cec517f4d5","created_at":"2026-09-01T10:37:58.293Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.297Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-371b125806cba7cec517f4d5: start","job_status":"start","queue_duration_s":2.861953,"scheduling_latency_s":2.86198}2495gitlab # [ 385.956914] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:01 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7bdb1025a7f20513f1ec42f289b6f901a9fe07ea HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 6432496gitlab # [ 385.961645] gitaly[932]: time="2026-09-01T10:38:01.175Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R6WY5R1JRTDCV7YCKE7J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:38:01.167" grpc.time_ms=7.827 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2497gitlab # [ 386.020049] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.201Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"371b125806cba7cec517f4d5","created_at":"2026-09-01T10:37:58.293Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.297Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-371b125806cba7cec517f4d5: done: 0.042257 sec","job_status":"done","queue_duration_s":2.861953,"scheduling_latency_s":2.86198,"redis_calls":10,"redis_duration_s":0.005723,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000679,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003419,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001625,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02619,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000415416,"duration_s":0.042257,"completed_at":"2026-09-01T10:38:01.201Z","load_balancing_strategy":"primary","db_duration_s":0.006628,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2498gitlab # [ 386.028779] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.203Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"1af4cbf90d9364a236746f9d","created_at":"2026-09-01T10:37:58.301Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.304Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-1af4cbf90d9364a236746f9d: start","job_status":"start","queue_duration_s":2.898986,"scheduling_latency_s":2.899008}2499gitlab # [ 386.084913] nixbot[3255]: {"ts": "2026-09-01T10:38:01.585Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/7bdb1025a7f20513f1ec42f289b6f901a9fe07ea \"HTTP/1.1 201 Created\""}2500gitlab # [ 386.085786] gitaly[932]: time="2026-09-01T10:38:01.260Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8R6ZSWSCKE4NNQG7PM813 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:38:01.254" grpc.time_ms=5.447 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2501gitlab # [ 386.087221] gitaly[932]: time="2026-09-01T10:38:01.330Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R71E2HEXT200PBXMS4JY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:38:01.319" pid=932 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc2502gitlab # [ 386.088161] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:01 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7bdb1025a7f20513f1ec42f289b6f901a9fe07ea HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1462503gitlab # [ 386.089203] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.258Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"1af4cbf90d9364a236746f9d","created_at":"2026-09-01T10:37:58.301Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.304Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-1af4cbf90d9364a236746f9d: done: 0.054881 sec","job_status":"done","queue_duration_s":2.898986,"scheduling_latency_s":2.899008,"redis_calls":14,"redis_duration_s":0.004608,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000811,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00179,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002007,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034368,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000510959,"duration_s":0.054881,"completed_at":"2026-09-01T10:38:01.258Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010077,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2504gitlab # [ 386.156127] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.260Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"75f5af6d6b5231ecfcfb6ab4","created_at":"2026-09-01T10:37:58.353Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.355Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-75f5af6d6b5231ecfcfb6ab4: start","job_status":"start","queue_duration_s":2.904806,"scheduling_latency_s":2.904829}2505gitlab # [ 386.157115] gitaly[932]: time="2026-09-01T10:38:01.330Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1163 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R71E2HEXT200PBXMS4JY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:38:01.319" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2506gitlab # [ 386.208470] nixbot[3255]: {"ts": "2026-09-01T10:38:01.721Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/7bdb1025a7f20513f1ec42f289b6f901a9fe07ea \"HTTP/1.1 201 Created\""}2507gitlab # [ 386.209584] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.289Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"75f5af6d6b5231ecfcfb6ab4","created_at":"2026-09-01T10:37:58.353Z","trace_propagation_headers":{"sentry-trace":"a0b78ec8a261437d95df6e1fcb6ce06f-211d9558329740d6","baggage":"sentry-trace_id=a0b78ec8a261437d95df6e1fcb6ce06f,sentry-release=Unknown"},"correlation_id":"01M1E8P2T0XK1F4TPTHNEXVHP1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.355Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-75f5af6d6b5231ecfcfb6ab4: done: 0.028842 sec","job_status":"done","queue_duration_s":2.904806,"scheduling_latency_s":2.904829,"redis_calls":8,"redis_duration_s":0.00329,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002849,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000441,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016717,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028842,"completed_at":"2026-09-01T10:38:01.289Z","load_balancing_strategy":"primary","db_duration_s":0.006419,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2508gitlab # [ 386.276409] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.290Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"ec40092c692e0b04ba33339a","created_at":"2026-09-01T10:37:58.425Z","trace_propagation_headers":{"sentry-trace":"fcb371ab68e545d89550b8c507a29ce6-4bf18f211cf94a78","baggage":"sentry-trace_id=fcb371ab68e545d89550b8c507a29ce6,sentry-release=Unknown"},"correlation_id":"01M1E8NT6V03XAPSJJAS0JE1RB","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.429Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-ec40092c692e0b04ba33339a: start","job_status":"start","queue_duration_s":2.861369,"scheduling_latency_s":2.861392}2509gitlab # [ 386.277904] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:01 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7bdb1025a7f20513f1ec42f289b6f901a9fe07ea HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1292510gitlab # [ 386.278817] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:01 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 442511gitlab # [ 386.279864] gitlab-workhorse[832]: time="2026-09-01T10:38:01Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672512gitlab # [ 386.280910] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:01 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 582513gitlab # [ 386.282445] gitaly[932]: time="2026-09-01T10:38:01.331Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1163 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R71E2HEXT200PBXMS4JY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:38:01.319" grpc.time_ms=12.073 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2514gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390'2515gitlab # [ 386.283422] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.337Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"ec40092c692e0b04ba33339a","created_at":"2026-09-01T10:37:58.425Z","trace_propagation_headers":{"sentry-trace":"fcb371ab68e545d89550b8c507a29ce6-4bf18f211cf94a78","baggage":"sentry-trace_id=fcb371ab68e545d89550b8c507a29ce6,sentry-release=Unknown"},"correlation_id":"01M1E8NT6V03XAPSJJAS0JE1RB","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:58.429Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-ec40092c692e0b04ba33339a: done: 0.047067 sec","job_status":"done","queue_duration_s":2.861369,"scheduling_latency_s":2.861392,"redis_calls":19,"redis_duration_s":0.004864,"redis_read_bytes":21,"redis_write_bytes":4289,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000828,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000451,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2260,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002827,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000758,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.007997665,"duration_s":0.047067,"completed_at":"2026-09-01T10:38:01.337Z","load_balancing_strategy":"primary","db_duration_s":0.005951,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2516gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390', in 0.04 seconds)2517[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'e8d9360f6d71fb57cb350a711a26cf7895d79390', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-09-01T10:38:01.350730Z', 'started_at': None, 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': {'position': 1, 'blocked_by': []}}]2518(finished: subtest: push delivers the webhook, in 72.68 seconds)2519subtest: eval and build post commit statuses2520gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses2521gitlab # [ 386.362297] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.339Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","03b53269eb86b5efa8774a37"],"class":"WebHooks::LogExecutionWorker","jid":"ae2b7621b729c234fb58dbb7","created_at":"2026-09-01T10:37:59.745Z","trace_propagation_headers":{"sentry-trace":"fcb371ab68e545d89550b8c507a29ce6-77dbd68749084791","baggage":"sentry-trace_id=fcb371ab68e545d89550b8c507a29ce6,sentry-release=Unknown"},"correlation_id":"01M1E8NT6V03XAPSJJAS0JE1RB","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c9ce356a285f4ba0843a71f3a9ae49528d262275152401dd88b412b3e78ed87e","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:59.751Z","job_size_bytes":2593,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-ae2b7621b729c234fb58dbb7: start","job_status":"start","queue_duration_s":1.588525,"scheduling_latency_s":1.588554}2522gitlab # [ 386.363906] gitaly[932]: time="2026-09-01T10:38:01.371Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=248 command.spawn_token_fork_ms=246 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8R6JPHX6ZNE72J76NGB1X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:38:11.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:38:01.121" grpc.time_ms=249.929 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2523gitlab # [ 386.424521] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.393Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","03b53269eb86b5efa8774a37"],"class":"WebHooks::LogExecutionWorker","jid":"ae2b7621b729c234fb58dbb7","created_at":"2026-09-01T10:37:59.745Z","trace_propagation_headers":{"sentry-trace":"fcb371ab68e545d89550b8c507a29ce6-77dbd68749084791","baggage":"sentry-trace_id=fcb371ab68e545d89550b8c507a29ce6,sentry-release=Unknown"},"correlation_id":"01M1E8NT6V03XAPSJJAS0JE1RB","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c9ce356a285f4ba0843a71f3a9ae49528d262275152401dd88b412b3e78ed87e","size_limiter":"validated","enqueued_at":"2026-09-01T10:37:59.751Z","job_size_bytes":2593,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-ae2b7621b729c234fb58dbb7: done: 0.053398 sec","job_status":"done","queue_duration_s":1.588525,"scheduling_latency_s":1.588554,"redis_calls":10,"redis_duration_s":0.005038,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000493,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00201,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002535,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031459,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000458438,"duration_s":0.053398,"completed_at":"2026-09-01T10:38:01.392Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005526,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2524gitlab # [ 386.431185] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.395Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","b45380ab5ddcae85a44e55df"],"class":"WebHooks::LogExecutionWorker","jid":"19ffbd49db211d96f3063ada","created_at":"2026-09-01T10:38:00.621Z","trace_propagation_headers":{"sentry-trace":"76d23fcdcb984775a6db1bdb1776e475-c4b30cd59aee4001","baggage":"sentry-trace_id=76d23fcdcb984775a6db1bdb1776e475,sentry-release=Unknown"},"correlation_id":"01M1E8P14TFMG1B28PPP39VJ7Z","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0cc8ef28f6d10ca92af52517f687f6fca4e327295a3d3c10cdfd13c73f59eeee","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:00.623Z","job_size_bytes":2591,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-19ffbd49db211d96f3063ada: start","job_status":"start","queue_duration_s":0.771381,"scheduling_latency_s":0.771407}2525gitlab # [ 386.506359] nixbot[3255]: INFO: ::ffff:127.0.0.1:40712 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 200 OK2526gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses, in 0.11 seconds)2527[]2528gitlab # [ 386.507856] gitaly[932]: time="2026-09-01T10:38:01.445Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R74G3VKRKNWVWH6NTY9M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:31.343" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:01.444" grpc.time_ms=0.571 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2529gitlab # [ 386.509043] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.422Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","b45380ab5ddcae85a44e55df"],"class":"WebHooks::LogExecutionWorker","jid":"19ffbd49db211d96f3063ada","created_at":"2026-09-01T10:38:00.621Z","trace_propagation_headers":{"sentry-trace":"76d23fcdcb984775a6db1bdb1776e475-c4b30cd59aee4001","baggage":"sentry-trace_id=76d23fcdcb984775a6db1bdb1776e475,sentry-release=Unknown"},"correlation_id":"01M1E8P14TFMG1B28PPP39VJ7Z","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0cc8ef28f6d10ca92af52517f687f6fca4e327295a3d3c10cdfd13c73f59eeee","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:00.623Z","job_size_bytes":2591,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-19ffbd49db211d96f3063ada: done: 0.027235 sec","job_status":"done","queue_duration_s":0.771381,"scheduling_latency_s":0.771407,"redis_calls":7,"redis_duration_s":0.00334,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002872,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000468,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027235,"completed_at":"2026-09-01T10:38:01.422Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003345,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2530gitlab # [ 386.532910] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.425Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"d52d913942223bb398341610","created_at":"2026-09-01T10:38:01.321Z","trace_propagation_headers":{"sentry-trace":"fcb371ab68e545d89550b8c507a29ce6-54683cce120e4548","baggage":"sentry-trace_id=fcb371ab68e545d89550b8c507a29ce6,sentry-release=Unknown"},"correlation_id":"01M1E8NT6V03XAPSJJAS0JE1RB","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.327Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-d52d913942223bb398341610: start","job_status":"start","queue_duration_s":0.09814,"scheduling_latency_s":0.098395}2531gitlab # [ 386.588979] gitaly[932]: time="2026-09-01T10:38:01.496Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R76Y52S2HDKADSG8EAAA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:31.495" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:01.496" grpc.time_ms=0.778 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2532gitlab # [ 386.589987] nixbot[3255]: {"ts": "2026-09-01T10:38:02.162Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e8d9360f6d71fb57cb350a711a26cf7895d79390 \"HTTP/1.1 201 Created\""}2533gitlab # [ 386.592883] nixbot[3255]: {"ts": "2026-09-01T10:38:02.171Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-e8d9360f6d71-3"}2534gitlab # [ 386.593860] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.512Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"d52d913942223bb398341610","created_at":"2026-09-01T10:38:01.321Z","trace_propagation_headers":{"sentry-trace":"fcb371ab68e545d89550b8c507a29ce6-54683cce120e4548","baggage":"sentry-trace_id=fcb371ab68e545d89550b8c507a29ce6,sentry-release=Unknown"},"correlation_id":"01M1E8NT6V03XAPSJJAS0JE1RB","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.327Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-d52d913942223bb398341610: done: 0.087244 sec","job_status":"done","queue_duration_s":0.09814,"scheduling_latency_s":0.098395,"redis_calls":11,"redis_duration_s":0.002774,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000551,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001727,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000496,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055477,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000439721,"duration_s":0.087244,"completed_at":"2026-09-01T10:38:01.512Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017833,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2535gitlab # [ 386.651904] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:01.513Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"320a7dee12051564a2b3e2f2","created_at":"2026-09-01T10:38:01.408Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-2bc79ff4bbbf44f2","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.418Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-320a7dee12051564a2b3e2f2: start","job_status":"start","queue_duration_s":0.095723,"scheduling_latency_s":0.095743}2536gitlab # [ 386.655984] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses HTTP/1.1" 200 2 "" "curl/8.21.0" 822537gitlab # [ 386.709463] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:02 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 8002538gitlab # [ 386.710368] gitaly[932]: time="2026-09-01T10:38:01.536Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8R76Y52S2HDKADSG8EAAA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:38:11.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:38:01.531" grpc.time_ms=4.631 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2539gitlab # [ 386.711042] gitaly[932]: time="2026-09-01T10:38:01.601Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8R6JPHX6ZNE72J76NGB1X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:38:01.600" grpc.time_ms=0.395 pid=932 span.kind=server system=grpc2540gitlab # [ 386.712579] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.016Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"2682ad6f811158691e9db8ee","created_at":"2026-09-01T10:38:02.014Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-33a0764caba442a4","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"320a7dee12051564a2b3e2f2","job_size_bytes":3,"pid":3276,"job_status":"deduplicated","message":"PipelineProcessWorker JID-2682ad6f811158691e9db8ee: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2541gitlab # [ 386.713675] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.018Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"320a7dee12051564a2b3e2f2","created_at":"2026-09-01T10:38:01.408Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-2bc79ff4bbbf44f2","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.418Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3276,"job_status":"rescheduled","message":"PipelineProcessWorker JID-320a7dee12051564a2b3e2f2: rescheduled"}2542gitlab # [ 386.771111] nixbot[3255]: {"ts": "2026-09-01T10:38:02.272Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-e8d9360f6d71-3", "argv0": "bwrap"}2543gitlab # [ 386.772245] gitaly[932]: time="2026-09-01T10:38:01.626Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8R6JPHX6ZNE72J76NGB1X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:38:01.625" grpc.time_ms=0.399 pid=932 span.kind=server system=grpc2544gitlab # [ 386.772830] gitaly[932]: time="2026-09-01T10:38:01.641Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R7BP1WFSX0Z72SZCDH8H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:31.641" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:01.641" grpc.time_ms=0.439 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2545gitlab # [ 386.773685] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.025Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"320a7dee12051564a2b3e2f2","created_at":"2026-09-01T10:38:01.408Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-2bc79ff4bbbf44f2","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.418Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-320a7dee12051564a2b3e2f2: done: 0.511603 sec","job_status":"done","queue_duration_s":0.095723,"scheduling_latency_s":0.095743,"gitaly_calls":3,"gitaly_duration_s":0.345102,"redis_calls":70,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.017946,"redis_read_bytes":227,"redis_write_bytes":18550,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000449,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002186,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.003901,"redis_queues_read_bytes":24,"redis_queues_write_bytes":12311,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.00789,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000374,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000626,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002071,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.006,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.116459,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.472697508,"duration_s":0.511603,"completed_at":"2026-09-01T10:38:02.025Z","load_balancing_strategy":"primary","db_duration_s":0.031504,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2546gitlab # [ 386.825698] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.030Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c84ccace36792d4808e814e2","created_at":"2026-09-01T10:38:01.565Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-33a0764caba442a4","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.567Z","job_size_bytes":23,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c84ccace36792d4808e814e2: start","job_status":"start","queue_duration_s":0.462749,"scheduling_latency_s":0.462833}2547gitlab # [ 386.885706] gitaly[932]: time="2026-09-01T10:38:01.653Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8R6JPHX6ZNE72J76NGB1X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:38:01.652" grpc.time_ms=0.521 pid=932 span.kind=server system=grpc2548gitlab # [ 386.886876] gitaly[932]: time="2026-09-01T10:38:01.676Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8R7BP1WFSX0Z72SZCDH8H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:38:11.141" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:38:01.671" grpc.time_ms=4.691 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2549gitlab # [ 386.889700] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.089Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c84ccace36792d4808e814e2","created_at":"2026-09-01T10:38:01.565Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-33a0764caba442a4","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.567Z","job_size_bytes":23,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c84ccace36792d4808e814e2: done: 0.059087 sec","job_status":"done","queue_duration_s":0.462749,"scheduling_latency_s":0.462833,"redis_calls":7,"redis_duration_s":0.002257,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001856,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000401,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035926,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.059087,"completed_at":"2026-09-01T10:38:02.089Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010835,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2550gitlab # [ 386.953108] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.093Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"dcf659d6096638af5b00727a","created_at":"2026-09-01T10:38:01.911Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-33a0764caba442a4","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.915Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-dcf659d6096638af5b00727a: start","job_status":"start","queue_duration_s":0.177479,"scheduling_latency_s":0.177519}2551gitlab # [ 386.954100] gitaly[932]: time="2026-09-01T10:38:01.786Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8R7GEWVSW99KE1KRCPVEF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:38:01.780" grpc.time_ms=5.98 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2552gitlab # [ 386.954837] gitaly[932]: time="2026-09-01T10:38:01.846Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R7HWMMYYD0D7DMF7KHE4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:38:01.841" pid=932 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2553gitlab # [ 386.961970] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.135Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"dcf659d6096638af5b00727a","created_at":"2026-09-01T10:38:01.911Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-33a0764caba442a4","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.915Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-dcf659d6096638af5b00727a: done: 0.042394 sec","job_status":"done","queue_duration_s":0.177479,"scheduling_latency_s":0.177519,"redis_calls":7,"redis_duration_s":0.005799,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004302,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001497,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021877,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.042394,"completed_at":"2026-09-01T10:38:02.135Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005328,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2554gitlab # [ 387.029106] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.137Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"4a787c84d1caac859a3e10b6","created_at":"2026-09-01T10:38:01.923Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-33a0764caba442a4","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.927Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-4a787c84d1caac859a3e10b6: start","job_status":"start","queue_duration_s":0.210465,"scheduling_latency_s":0.210498}2555gitlab # [ 387.030499] gitaly[932]: time="2026-09-01T10:38:01.846Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R7HWMMYYD0D7DMF7KHE4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:38:01.841" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2556gitlab # [ 387.089287] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.167Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"4a787c84d1caac859a3e10b6","created_at":"2026-09-01T10:38:01.923Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-33a0764caba442a4","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.927Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-4a787c84d1caac859a3e10b6: done: 0.029535 sec","job_status":"done","queue_duration_s":0.210465,"scheduling_latency_s":0.210498,"redis_calls":7,"redis_duration_s":0.001981,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001551,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00043,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029535,"completed_at":"2026-09-01T10:38:02.167Z","load_balancing_strategy":"primary","db_duration_s":0.004845,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2557gitlab # [ 387.097072] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.169Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"a0b6c0267a876ad0cb030884","created_at":"2026-09-01T10:38:01.935Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-33a0764caba442a4","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.937Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-a0b6c0267a876ad0cb030884: start","job_status":"start","queue_duration_s":0.231629,"scheduling_latency_s":0.23166}2558gitlab # [ 387.115411] gitaly[932]: time="2026-09-01T10:38:01.847Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R7HWMMYYD0D7DMF7KHE4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:38:01.841" grpc.time_ms=5.517 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2559gitlab # [ 387.167970] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.206Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"a0b6c0267a876ad0cb030884","created_at":"2026-09-01T10:38:01.935Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-33a0764caba442a4","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.937Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-a0b6c0267a876ad0cb030884: done: 0.036992 sec","job_status":"done","queue_duration_s":0.231629,"scheduling_latency_s":0.23166,"redis_calls":10,"redis_duration_s":0.002286,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001192,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00061,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000484,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022171,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036992,"completed_at":"2026-09-01T10:38:02.206Z","load_balancing_strategy":"primary","db_duration_s":0.007115,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2560gitlab # [ 387.183640] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.208Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"7bea97b0285f99876b645f21","created_at":"2026-09-01T10:38:01.938Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-33a0764caba442a4","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.939Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-7bea97b0285f99876b645f21: start","job_status":"start","queue_duration_s":0.268332,"scheduling_latency_s":0.268365}2561gitlab # [ 387.239528] gitaly[932]: time="2026-09-01T10:38:01.900Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=443 command.spawn_token_fork_ms=441 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8R74G3VKRKNWVWH6NTY9M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-01T10:38:31.354" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:38:01.455" grpc.time_ms=444.966 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2562gitlab # [ 387.240780] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.233Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"7bea97b0285f99876b645f21","created_at":"2026-09-01T10:38:01.938Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-33a0764caba442a4","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.939Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-7bea97b0285f99876b645f21: done: 0.025281 sec","job_status":"done","queue_duration_s":0.268332,"scheduling_latency_s":0.268365,"redis_calls":7,"redis_duration_s":0.0015459999999999998,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001133,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000413,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016256,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025281,"completed_at":"2026-09-01T10:38:02.233Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003585,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2563gitlab # [ 387.308756] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.235Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"1eedc8063989f0171c9c5cda","created_at":"2026-09-01T10:38:01.940Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-33a0764caba442a4","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.943Z","job_size_bytes":19,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-1eedc8063989f0171c9c5cda: start","job_status":"start","queue_duration_s":0.292349,"scheduling_latency_s":0.292375}2564gitlab # [ 387.310061] gitaly[932]: time="2026-09-01T10:38:01.905Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R6JPHX6ZNE72J76NGB1X diskcache=c243e35a-7b92-4f0e-ad14-6d294b51a292 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T10:38:31.472" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:38:01.573" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2565gitlab # [ 387.311097] gitaly[932]: time="2026-09-01T10:38:01.905Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=68 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=827392 command.minflt=7141 command.oublock=0 command.real_time_ms=325 command.spawn_token_fork_ms=0 command.system_time_ms=36 command.user_time_ms=32 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R6JPHX6ZNE72J76NGB1X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T10:38:31.472" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:38:01.573" grpc.time_ms=332.167 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2566gitlab # [ 387.365505] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.263Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"1eedc8063989f0171c9c5cda","created_at":"2026-09-01T10:38:01.940Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-33a0764caba442a4","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.943Z","job_size_bytes":19,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-1eedc8063989f0171c9c5cda: done: 0.028101 sec","job_status":"done","queue_duration_s":0.292349,"scheduling_latency_s":0.292375,"redis_calls":8,"redis_duration_s":0.0031079999999999997,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002662,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000446,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016949,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028101,"completed_at":"2026-09-01T10:38:02.263Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005262,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2567gitlab # [ 387.367302] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.265Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"60c94387e09785e05d7e091c","created_at":"2026-09-01T10:38:01.944Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-33a0764caba442a4","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.947Z","job_size_bytes":66,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-60c94387e09785e05d7e091c: start","job_status":"start","queue_duration_s":0.318074,"scheduling_latency_s":0.318098}2568gitlab # [ 387.388054] gitaly[932]: time="2026-09-01T10:38:01.926Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=10 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8R74G3VKRKNWVWH6NTY9M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:38:11.220" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:38:01.911" grpc.time_ms=14.929 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2569gitlab # [ 387.440922] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.291Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"60c94387e09785e05d7e091c","created_at":"2026-09-01T10:38:01.944Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-33a0764caba442a4","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.947Z","job_size_bytes":66,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-60c94387e09785e05d7e091c: done: 0.025442 sec","job_status":"done","queue_duration_s":0.318074,"scheduling_latency_s":0.318098,"redis_calls":7,"redis_duration_s":0.0018770000000000002,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001324,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000553,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025442,"completed_at":"2026-09-01T10:38:02.290Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002717,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2570gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses2571gitlab # [ 387.457276] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.292Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"1bf753ba944c99dab6687ec3","created_at":"2026-09-01T10:38:01.949Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-33a0764caba442a4","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.950Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-1bf753ba944c99dab6687ec3: start","job_status":"start","queue_duration_s":0.341972,"scheduling_latency_s":0.341995}2572gitlab # [ 387.478051] gitaly[932]: time="2026-09-01T10:38:01.948Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R74G3VKRKNWVWH6NTY9M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-01T10:38:11.257" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:38:01.948" grpc.time_ms=0.224 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2573gitlab # [ 387.491511] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.318Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"1bf753ba944c99dab6687ec3","created_at":"2026-09-01T10:38:01.949Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-33a0764caba442a4","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.950Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-1bf753ba944c99dab6687ec3: done: 0.026144 sec","job_status":"done","queue_duration_s":0.341972,"scheduling_latency_s":0.341995,"redis_calls":7,"redis_duration_s":0.0016510000000000001,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001245,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000406,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026144,"completed_at":"2026-09-01T10:38:02.318Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00487,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2574gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses, in 0.09 seconds)2575[{'id': 5, 'sha': 'e8d9360f6d71fb57cb350a711a26cf7895d79390', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-01T10:38:02.137Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2576gitlab # [ 387.546384] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.320Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"f6a2d46f684dffc0e0c5003b","created_at":"2026-09-01T10:38:01.951Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-33a0764caba442a4","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.952Z","job_size_bytes":6,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-f6a2d46f684dffc0e0c5003b: start","job_status":"start","queue_duration_s":0.368185,"scheduling_latency_s":0.368207}2577gitlab # [ 387.610413] gitaly[932]: time="2026-09-01T10:38:01.956Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R74G3VKRKNWVWH6NTY9M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-01T10:38:11.261" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:38:01.952" grpc.time_ms=4.555 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2578gitlab # [ 387.611372] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:03 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 592579gitlab # [ 387.612263] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.336Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"f6a2d46f684dffc0e0c5003b","created_at":"2026-09-01T10:38:01.951Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-33a0764caba442a4","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.952Z","job_size_bytes":6,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-f6a2d46f684dffc0e0c5003b: done: 0.015567 sec","job_status":"done","queue_duration_s":0.368185,"scheduling_latency_s":0.368207,"redis_calls":7,"redis_duration_s":0.001487,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001021,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000466,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015567,"completed_at":"2026-09-01T10:38:02.336Z","load_balancing_strategy":"primary","db_duration_s":0.000464,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2580gitlab # [ 387.668769] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.338Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"606f00c6c10744fabcb7150c","created_at":"2026-09-01T10:38:01.953Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-33a0764caba442a4","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.954Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-606f00c6c10744fabcb7150c: start","job_status":"start","queue_duration_s":0.383539,"scheduling_latency_s":0.383562}2581gitlab # [ 387.669939] gitaly[932]: time="2026-09-01T10:38:01.965Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8R6JPHX6ZNE72J76NGB1X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-01T10:38:31.865" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:01.965" grpc.time_ms=0.581 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2582gitlab # [ 387.670714] gitaly[932]: time="2026-09-01T10:38:01.999Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8R74G3VKRKNWVWH6NTY9M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:38:01.998" grpc.time_ms=0.348 pid=932 span.kind=server system=grpc2583gitlab # [ 387.681114] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.360Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"606f00c6c10744fabcb7150c","created_at":"2026-09-01T10:38:01.953Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-33a0764caba442a4","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:01.954Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-606f00c6c10744fabcb7150c: done: 0.022671 sec","job_status":"done","queue_duration_s":0.383539,"scheduling_latency_s":0.383562,"redis_calls":7,"redis_duration_s":0.00152,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001131,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000389,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014872,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022671,"completed_at":"2026-09-01T10:38:02.360Z","load_balancing_strategy":"primary","db_duration_s":0.003212,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2584gitlab # [ 387.740114] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.362Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"d690435ceaff794bf1933eeb","created_at":"2026-09-01T10:38:02.019Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-33a0764caba442a4","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.020Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d690435ceaff794bf1933eeb: start","job_status":"start","queue_duration_s":0.3421,"scheduling_latency_s":0.342124}2585gitlab # [ 387.749162] gitaly[932]: time="2026-09-01T10:38:02.003Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R6JPHX6ZNE72J76NGB1X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:32.002" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:02.002" grpc.time_ms=0.633 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2586gitlab # [ 387.803125] gitaly[932]: time="2026-09-01T10:38:02.027Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8R74G3VKRKNWVWH6NTY9M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:38:02.025" grpc.time_ms=0.973 pid=932 span.kind=server system=grpc2587gitlab # [ 387.804441] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.552Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"d690435ceaff794bf1933eeb","created_at":"2026-09-01T10:38:02.019Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-33a0764caba442a4","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.020Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d690435ceaff794bf1933eeb: done: 0.190168 sec","job_status":"done","queue_duration_s":0.3421,"scheduling_latency_s":0.342124,"gitaly_calls":3,"gitaly_duration_s":0.068286,"redis_calls":63,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.011082,"redis_read_bytes":144,"redis_write_bytes":16712,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000436,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000817,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":20,"redis_queues_duration_s":0.002376,"redis_queues_read_bytes":24,"redis_queues_write_bytes":11218,"redis_queues_metadata_calls":18,"redis_queues_metadata_duration_s":0.004631,"redis_queues_metadata_read_bytes":28,"redis_queues_metadata_write_bytes":2886,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000216,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000522,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001648,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.092844,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.160853964,"duration_s":0.190168,"completed_at":"2026-09-01T10:38:02.552Z","load_balancing_strategy":"primary","db_duration_s":0.022151,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2588gitlab # [ 387.867821] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.554Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"c6ce64917ecd275139e005b3","created_at":"2026-09-01T10:38:02.051Z","trace_propagation_headers":{"sentry-trace":"7529e01ab561451283ad31267162e00d-3b6924c105484323","baggage":"sentry-trace_id=7529e01ab561451283ad31267162e00d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8R74G3VKRKNWVWH6NTY9M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.054Z","job_size_bytes":65,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-c6ce64917ecd275139e005b3: start","job_status":"start","queue_duration_s":0.499218,"scheduling_latency_s":0.499239}2589gitlab # [ 387.868911] gitaly[932]: time="2026-09-01T10:38:02.042Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R74G3VKRKNWVWH6NTY9M diskcache=b9920a4e-e2a8-4cbf-8700-8fba89b20ae3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-01T10:38:11.268" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:38:01.959" pid=932 ref=refs/keep-around/e8d9360f6d71fb57cb350a711a26cf7895d79390 remote_ip="2001:db8:1::1" revision=e8d9360f6d71fb57cb350a711a26cf7895d79390 span.kind=server system=grpc username=root2590gitlab # [ 387.870592] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.579Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"c6ce64917ecd275139e005b3","created_at":"2026-09-01T10:38:02.051Z","trace_propagation_headers":{"sentry-trace":"7529e01ab561451283ad31267162e00d-3b6924c105484323","baggage":"sentry-trace_id=7529e01ab561451283ad31267162e00d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8R74G3VKRKNWVWH6NTY9M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.054Z","job_size_bytes":65,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-c6ce64917ecd275139e005b3: done: 0.025264 sec","job_status":"done","queue_duration_s":0.499218,"scheduling_latency_s":0.499239,"redis_calls":7,"redis_duration_s":0.001551,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001144,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000407,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017432,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025264,"completed_at":"2026-09-01T10:38:02.579Z","load_balancing_strategy":"primary","db_duration_s":0.003614,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2591gitlab # [ 387.935053] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.580Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"d6d1caacc55265380d1606d0","created_at":"2026-09-01T10:38:02.055Z","trace_propagation_headers":{"sentry-trace":"7529e01ab561451283ad31267162e00d-3b6924c105484323","baggage":"sentry-trace_id=7529e01ab561451283ad31267162e00d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8R74G3VKRKNWVWH6NTY9M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.058Z","job_size_bytes":65,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-d6d1caacc55265380d1606d0: start","job_status":"start","queue_duration_s":0.522638,"scheduling_latency_s":0.522659}2592gitlab # [ 387.941097] gitaly[932]: time="2026-09-01T10:38:02.042Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=51 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5095 command.oublock=8 command.real_time_ms=68 command.spawn_token_fork_ms=0 command.system_time_ms=33 command.user_time_ms=18 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R74G3VKRKNWVWH6NTY9M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-01T10:38:11.268" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T10:38:01.959" grpc.time_ms=82.727 pid=932 ref=refs/keep-around/e8d9360f6d71fb57cb350a711a26cf7895d79390 remote_ip="2001:db8:1::1" revision=e8d9360f6d71fb57cb350a711a26cf7895d79390 span.kind=server system=grpc username=root2593gitlab # [ 387.959163] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.625Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"d6d1caacc55265380d1606d0","created_at":"2026-09-01T10:38:02.055Z","trace_propagation_headers":{"sentry-trace":"7529e01ab561451283ad31267162e00d-3b6924c105484323","baggage":"sentry-trace_id=7529e01ab561451283ad31267162e00d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8R74G3VKRKNWVWH6NTY9M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.058Z","job_size_bytes":65,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-d6d1caacc55265380d1606d0: done: 0.044399 sec","job_status":"done","queue_duration_s":0.522638,"scheduling_latency_s":0.522659,"redis_calls":10,"redis_duration_s":0.002015,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001014,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001001,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044399,"completed_at":"2026-09-01T10:38:02.625Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009191,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2594gitlab # [ 388.014871] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.626Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"5b8d9dc09f04404d17690953","created_at":"2026-09-01T10:38:02.142Z","trace_propagation_headers":{"sentry-trace":"7529e01ab561451283ad31267162e00d-3b6924c105484323","baggage":"sentry-trace_id=7529e01ab561451283ad31267162e00d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8R74G3VKRKNWVWH6NTY9M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.146Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5b8d9dc09f04404d17690953: start","job_status":"start","queue_duration_s":0.480143,"scheduling_latency_s":0.480165}2595gitlab # [ 388.029099] gitaly[932]: time="2026-09-01T10:38:02.049Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R74G3VKRKNWVWH6NTY9M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:32.047" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:02.048" grpc.time_ms=1.192 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2596gitlab # [ 388.043861] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.717Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"5b8d9dc09f04404d17690953","created_at":"2026-09-01T10:38:02.142Z","trace_propagation_headers":{"sentry-trace":"7529e01ab561451283ad31267162e00d-3b6924c105484323","baggage":"sentry-trace_id=7529e01ab561451283ad31267162e00d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E8R74G3VKRKNWVWH6NTY9M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.146Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5b8d9dc09f04404d17690953: done: 0.091374 sec","job_status":"done","queue_duration_s":0.480143,"scheduling_latency_s":0.480165,"gitaly_calls":2,"gitaly_duration_s":0.004794,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.005874,"redis_read_bytes":80,"redis_write_bytes":5121,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000894,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.00086,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000301,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001411,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00024,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000338,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000936,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.065114,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.063881862,"duration_s":0.091374,"completed_at":"2026-09-01T10:38:02.717Z","load_balancing_strategy":"primary","db_duration_s":0.017944,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2597gitlab # [ 388.103789] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.720Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"7386f26029ce639deaf73364","created_at":"2026-09-01T10:38:02.408Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-9adc1ebf96c34638","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.410Z","job_size_bytes":23,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-7386f26029ce639deaf73364: start","job_status":"start","queue_duration_s":0.309949,"scheduling_latency_s":0.309969}2598gitlab # [ 388.161051] gitaly[932]: time="2026-09-01T10:38:02.086Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R7SF2FZ97A09X51WD9JX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:32.085" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:02.085" grpc.time_ms=0.543 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2599gitlab # [ 388.161867] gitaly[932]: time="2026-09-01T10:38:02.435Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8R6JPHX6ZNE72J76NGB1X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:38:02.434" grpc.time_ms=0.417 pid=932 span.kind=server system=grpc2600gitlab # [ 388.162870] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.750Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"7386f26029ce639deaf73364","created_at":"2026-09-01T10:38:02.408Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-9adc1ebf96c34638","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.410Z","job_size_bytes":23,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-7386f26029ce639deaf73364: done: 0.030647 sec","job_status":"done","queue_duration_s":0.309949,"scheduling_latency_s":0.309969,"redis_calls":7,"redis_duration_s":0.001351,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00099,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000361,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021828,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030647,"completed_at":"2026-09-01T10:38:02.750Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005773,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2601gitlab # [ 388.240205] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.752Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"5daddb4e5e3603c8a997dbcb","created_at":"2026-09-01T10:38:02.480Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-9adc1ebf96c34638","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.482Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-5daddb4e5e3603c8a997dbcb: start","job_status":"start","queue_duration_s":0.269695,"scheduling_latency_s":0.269727}2602gitlab # [ 388.241195] gitaly[932]: time="2026-09-01T10:38:02.453Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8R6JPHX6ZNE72J76NGB1X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:38:02.453" grpc.time_ms=0.391 pid=932 span.kind=server system=grpc2603gitlab # [ 388.242210] gitaly[932]: time="2026-09-01T10:38:02.469Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E8R6JPHX6ZNE72J76NGB1X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T10:38:02.469" grpc.time_ms=0.342 pid=932 span.kind=server system=grpc2604gitlab # [ 388.243199] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.776Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"5daddb4e5e3603c8a997dbcb","created_at":"2026-09-01T10:38:02.480Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-9adc1ebf96c34638","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.482Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-5daddb4e5e3603c8a997dbcb: done: 0.024398 sec","job_status":"done","queue_duration_s":0.269695,"scheduling_latency_s":0.269727,"redis_calls":7,"redis_duration_s":0.001568,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00118,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000388,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016165,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024398,"completed_at":"2026-09-01T10:38:02.776Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003538,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2605gitlab # [ 388.295646] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.778Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"42c28106074e5c6561763a4b","created_at":"2026-09-01T10:38:02.490Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-9adc1ebf96c34638","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.493Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-42c28106074e5c6561763a4b: start","job_status":"start","queue_duration_s":0.284687,"scheduling_latency_s":0.284714}2606gitlab # [ 388.311199] gitaly[932]: time="2026-09-01T10:38:02.472Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R6JPHX6ZNE72J76NGB1X forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T10:38:32.314" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:38:02.414" operations="[repack_refs]" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2607gitlab # [ 388.362319] gitaly[932]: time="2026-09-01T10:38:02.473Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R6JPHX6ZNE72J76NGB1X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T10:38:32.314" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:38:02.414" pid=932 remote_ip="2001:db8:1::1" repository_info="{false {6 703 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-09-01 10:36:40.186417932 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root2608gitlab # [ 388.363386] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.801Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"42c28106074e5c6561763a4b","created_at":"2026-09-01T10:38:02.490Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-9adc1ebf96c34638","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.493Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-42c28106074e5c6561763a4b: done: 0.023719 sec","job_status":"done","queue_duration_s":0.284687,"scheduling_latency_s":0.284714,"redis_calls":7,"redis_duration_s":0.001503,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001137,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000366,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023719,"completed_at":"2026-09-01T10:38:02.801Z","load_balancing_strategy":"primary","db_duration_s":0.003849,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2609gitlab # [ 388.416654] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.803Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"ce048ea43fb6dcc9635a973f","created_at":"2026-09-01T10:38:02.499Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-9adc1ebf96c34638","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.500Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-ce048ea43fb6dcc9635a973f: start","job_status":"start","queue_duration_s":0.302467,"scheduling_latency_s":0.302488}2610gitlab # [ 388.417517] gitaly[932]: time="2026-09-01T10:38:02.475Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R6JPHX6ZNE72J76NGB1X diskcache=7377d613-843a-47e8-8300-014f25037ae2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T10:38:32.314" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:38:02.414" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2611gitlab # [ 388.418327] gitaly[932]: time="2026-09-01T10:38:02.475Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=66 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7283 command.oublock=0 command.real_time_ms=57 command.spawn_token_fork_ms=0 command.system_time_ms=48 command.user_time_ms=18 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R6JPHX6ZNE72J76NGB1X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T10:38:32.314" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:38:02.414" grpc.time_ms=60.891 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2612gitlab # [ 388.430080] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.827Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"ce048ea43fb6dcc9635a973f","created_at":"2026-09-01T10:38:02.499Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-9adc1ebf96c34638","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.500Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-ce048ea43fb6dcc9635a973f: done: 0.02446 sec","job_status":"done","queue_duration_s":0.302467,"scheduling_latency_s":0.302488,"redis_calls":10,"redis_duration_s":0.0021279999999999997,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001089,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000658,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000381,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016564,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02446,"completed_at":"2026-09-01T10:38:02.827Z","load_balancing_strategy":"primary","db_duration_s":0.003739,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2613gitlab # [ 388.494998] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.829Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"223d154e9093407173c30946","created_at":"2026-09-01T10:38:02.501Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-9adc1ebf96c34638","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.502Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-223d154e9093407173c30946: start","job_status":"start","queue_duration_s":0.326777,"scheduling_latency_s":0.326798}2614gitlab # [ 388.500756] gitaly[932]: time="2026-09-01T10:38:02.488Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R6JPHX6ZNE72J76NGB1X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T10:38:32.314" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:38:02.414" pid=932 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2615gitlab # [ 388.516800] gitaly[932]: time="2026-09-01T10:38:02.489Z" level=info msg="optimized repository" clean_stale_data_time=14.140624ms clean_work_trees_time="137.727µs" commit_graph_time="6.426µs" correlation_id=01M1E8R6JPHX6ZNE72J76NGB1X gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="6.984µs" pid=932 prune_time="6.705µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="8.66µs" storage_name=default system=housekeeping total_time=14.554644ms2616gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses2617gitlab # [ 388.523830] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.850Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"223d154e9093407173c30946","created_at":"2026-09-01T10:38:02.501Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-9adc1ebf96c34638","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.502Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-223d154e9093407173c30946: done: 0.021265 sec","job_status":"done","queue_duration_s":0.326777,"scheduling_latency_s":0.326798,"redis_calls":7,"redis_duration_s":0.001485,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001081,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000404,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01429,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021265,"completed_at":"2026-09-01T10:38:02.850Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0032,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2618gitlab # [ 388.575159] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.851Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"31f7179b93f3c9e6d5866425","created_at":"2026-09-01T10:38:02.502Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-9adc1ebf96c34638","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.504Z","job_size_bytes":19,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-31f7179b93f3c9e6d5866425: start","job_status":"start","queue_duration_s":0.347433,"scheduling_latency_s":0.347457}2619gitlab # [ 388.592281] gitaly[932]: time="2026-09-01T10:38:02.489Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R6JPHX6ZNE72J76NGB1X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T10:38:32.314" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:38:02.414" pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2620gitlab # [ 388.603470] gitaly[932]: time="2026-09-01T10:38:02.518Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8R6JPHX6ZNE72J76NGB1X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-01T10:38:32.417" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:02.517" grpc.time_ms=0.678 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2621gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses, in 0.08 seconds)2622[{'id': 5, 'sha': 'e8d9360f6d71fb57cb350a711a26cf7895d79390', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-01T10:38:02.137Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2623gitlab # [ 388.618981] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.873Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"31f7179b93f3c9e6d5866425","created_at":"2026-09-01T10:38:02.502Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-9adc1ebf96c34638","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.504Z","job_size_bytes":19,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-31f7179b93f3c9e6d5866425: done: 0.021271 sec","job_status":"done","queue_duration_s":0.347433,"scheduling_latency_s":0.347457,"redis_calls":8,"redis_duration_s":0.0017980000000000001,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001392,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000406,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014374,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021271,"completed_at":"2026-09-01T10:38:02.873Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002617,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2624gitlab # [ 388.676755] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.874Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"c44c171fc1249e010c200eeb","created_at":"2026-09-01T10:38:02.505Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-9adc1ebf96c34638","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.506Z","job_size_bytes":66,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-c44c171fc1249e010c200eeb: start","job_status":"start","queue_duration_s":0.368322,"scheduling_latency_s":0.368345}2625gitlab # [ 388.696053] gitaly[932]: time="2026-09-01T10:38:02.541Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R6JPHX6ZNE72J76NGB1X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:32.440" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:02.540" grpc.time_ms=0.442 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2626gitlab # [ 388.713501] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:04 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 542627gitlab # [ 388.715967] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.894Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"c44c171fc1249e010c200eeb","created_at":"2026-09-01T10:38:02.505Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-9adc1ebf96c34638","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.506Z","job_size_bytes":66,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-c44c171fc1249e010c200eeb: done: 0.020213 sec","job_status":"done","queue_duration_s":0.368322,"scheduling_latency_s":0.368345,"redis_calls":7,"redis_duration_s":0.001785,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001352,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000433,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013241,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020213,"completed_at":"2026-09-01T10:38:02.894Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002705,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2628gitlab # [ 388.799314] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.896Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"d6d59e3af594e463e9d57456","created_at":"2026-09-01T10:38:02.506Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-9adc1ebf96c34638","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.507Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-d6d59e3af594e463e9d57456: start","job_status":"start","queue_duration_s":0.388564,"scheduling_latency_s":0.38859}2629gitlab # [ 388.800341] gitaly[932]: time="2026-09-01T10:38:02.685Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8R74G3VKRKNWVWH6NTY9M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-01T10:38:32.584" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:02.684" grpc.time_ms=0.65 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2630gitlab # [ 388.801924] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.917Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"d6d59e3af594e463e9d57456","created_at":"2026-09-01T10:38:02.506Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-9adc1ebf96c34638","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.507Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-d6d59e3af594e463e9d57456: done: 0.021095 sec","job_status":"done","queue_duration_s":0.388564,"scheduling_latency_s":0.38859,"redis_calls":7,"redis_duration_s":0.001531,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001104,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000427,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021095,"completed_at":"2026-09-01T10:38:02.917Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003923,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2631gitlab # [ 388.888132] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.918Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"e2580544d3d3449230b6445b","created_at":"2026-09-01T10:38:02.508Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-9adc1ebf96c34638","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.509Z","job_size_bytes":6,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-e2580544d3d3449230b6445b: start","job_status":"start","queue_duration_s":0.409612,"scheduling_latency_s":0.409633}2632gitlab # [ 388.889144] gitaly[932]: time="2026-09-01T10:38:02.709Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R74G3VKRKNWVWH6NTY9M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:32.608" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:02.709" grpc.time_ms=0.482 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2633gitlab # [ 388.890130] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.931Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"e2580544d3d3449230b6445b","created_at":"2026-09-01T10:38:02.508Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-9adc1ebf96c34638","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.509Z","job_size_bytes":6,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-e2580544d3d3449230b6445b: done: 0.012672 sec","job_status":"done","queue_duration_s":0.409612,"scheduling_latency_s":0.409633,"redis_calls":7,"redis_duration_s":0.001408,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000992,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000416,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008221,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012672,"completed_at":"2026-09-01T10:38:02.931Z","load_balancing_strategy":"primary","db_duration_s":0.000443,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2634gitlab # [ 388.963760] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.932Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"d49bb258ebae9e582b63f356","created_at":"2026-09-01T10:38:02.509Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-9adc1ebf96c34638","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.510Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-d49bb258ebae9e582b63f356: start","job_status":"start","queue_duration_s":0.422031,"scheduling_latency_s":0.422051}2635gitlab # [ 388.964879] gitaly[932]: time="2026-09-01T10:38:03.181Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R8W9X1GXTA717KK7TMG5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:33.179" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:03.180" grpc.time_ms=0.68 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2636gitlab # [ 388.967883] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.953Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"d49bb258ebae9e582b63f356","created_at":"2026-09-01T10:38:02.509Z","trace_propagation_headers":{"sentry-trace":"165c59640c844dc8a561d1f7ba449f3f-9adc1ebf96c34638","baggage":"sentry-trace_id=165c59640c844dc8a561d1f7ba449f3f,sentry-release=Unknown"},"correlation_id":"01M1E8R6JPHX6ZNE72J76NGB1X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.510Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-d49bb258ebae9e582b63f356: done: 0.020357 sec","job_status":"done","queue_duration_s":0.422031,"scheduling_latency_s":0.422051,"redis_calls":7,"redis_duration_s":0.001767,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001341,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000426,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020357,"completed_at":"2026-09-01T10:38:02.953Z","load_balancing_strategy":"primary","db_duration_s":0.003119,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2637gitlab # [ 389.034427] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.954Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"74eaaa5db6592648d04aab6e","created_at":"2026-09-01T10:38:02.672Z","trace_propagation_headers":{"sentry-trace":"7529e01ab561451283ad31267162e00d-f94ae2e439fc4ff8","baggage":"sentry-trace_id=7529e01ab561451283ad31267162e00d,sentry-release=Unknown"},"correlation_id":"01M1E8R74G3VKRKNWVWH6NTY9M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.673Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-74eaaa5db6592648d04aab6e: start","job_status":"start","queue_duration_s":0.281203,"scheduling_latency_s":0.281229}2638gitlab # [ 389.035426] gitaly[932]: time="2026-09-01T10:38:04.269Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8R9Y86EK8HBSR943Y7A18 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:34.268" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:04.268" grpc.time_ms=0.626 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2639gitlab # [ 389.095981] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:02.985Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"74eaaa5db6592648d04aab6e","created_at":"2026-09-01T10:38:02.672Z","trace_propagation_headers":{"sentry-trace":"7529e01ab561451283ad31267162e00d-f94ae2e439fc4ff8","baggage":"sentry-trace_id=7529e01ab561451283ad31267162e00d,sentry-release=Unknown"},"correlation_id":"01M1E8R74G3VKRKNWVWH6NTY9M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:02.673Z","job_size_bytes":3,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-74eaaa5db6592648d04aab6e: done: 0.030869 sec","job_status":"done","queue_duration_s":0.281203,"scheduling_latency_s":0.281229,"redis_calls":7,"redis_duration_s":0.001464,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001076,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000388,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022776,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030869,"completed_at":"2026-09-01T10:38:02.985Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00388,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2640gitlab # [ 389.215353] nix-daemon[3502]: accepted connection from pid 4172, user nixbot2641gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses2642gitlab # [ 389.749727] gitaly[932]: time="2026-09-01T10:38:05.350Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8RB02Z0T52FV9E9K8YQTZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:35.348" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:05.349" grpc.time_ms=0.634 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2643gitlab # [ 389.767667] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:05 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 532644gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses, in 0.08 seconds)2645[{'id': 5, 'sha': 'e8d9360f6d71fb57cb350a711a26cf7895d79390', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-01T10:38:02.137Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2646gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses2647gitlab # [ 390.830914] gitaly[932]: time="2026-09-01T10:38:06.431Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8RC1KJC77M2X6FSJS8E6F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:36.430" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:06.430" grpc.time_ms=0.543 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2648gitlab # [ 390.868831] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:06 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 812649gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses, in 0.10 seconds)2650[{'id': 5, 'sha': 'e8d9360f6d71fb57cb350a711a26cf7895d79390', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-01T10:38:02.137Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2651gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses2652gitlab # [ 391.921119] gitaly[932]: time="2026-09-01T10:38:07.521Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8RD3Z0ZD733P52K7HGMX2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:37.520" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:07.520" grpc.time_ms=0.712 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2653gitlab # [ 391.937947] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:07 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 502654gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses, in 0.07 seconds)2655[{'id': 5, 'sha': 'e8d9360f6d71fb57cb350a711a26cf7895d79390', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-01T10:38:02.137Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2656gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses2657gitlab # [ 393.000475] gitaly[932]: time="2026-09-01T10:38:08.600Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8RE5DCQWSNWQS65XRT0NE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:38.599" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:08.600" grpc.time_ms=0.612 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2658gitlab # [ 393.024258] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:08 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 672659gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses, in 0.09 seconds)2660[{'id': 5, 'sha': 'e8d9360f6d71fb57cb350a711a26cf7895d79390', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-01T10:38:02.137Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2661gitlab # [ 393.352997] nix-daemon[3502]: accepted connection from pid 4207, user nixbot2662gitlab # [ 393.362125] nix-daemon[3502]: accepted connection from pid 4172, user nixbot2663gitlab # [ 393.406960] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:09.007Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"2f465a7673c4d7ae0db0ceee","created_at":"2026-09-01T10:38:08.994Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9fbd1830eee5d4e6151593936acc4ea4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:09.004Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-2f465a7673c4d7ae0db0ceee: start","job_status":"start","queue_duration_s":0.001944,"scheduling_latency_s":0.001983}2664gitlab # [ 393.442047] gitaly[932]: time="2026-09-01T10:38:09.042Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8REJXQ8GD00PXJG1VX82T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:39.041" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:09.041" grpc.time_ms=0.613 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2665gitlab # [ 393.444827] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:09.043Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"2f465a7673c4d7ae0db0ceee","created_at":"2026-09-01T10:38:08.994Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9fbd1830eee5d4e6151593936acc4ea4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:09.004Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-2f465a7673c4d7ae0db0ceee: done: 0.036361 sec","job_status":"done","queue_duration_s":0.001944,"scheduling_latency_s":0.001983,"redis_calls":11,"redis_duration_s":0.008959,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001171,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003124,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004664,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036361,"completed_at":"2026-09-01T10:38:09.043Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00103,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2666gitlab # [ 393.508047] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:09.065Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"0afe284c171707b6a926ce8c","created_at":"2026-09-01T10:38:09.035Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8d5c4eb800c01f9714aa6c6742032b7","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:09.048Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-0afe284c171707b6a926ce8c: start","job_status":"start","queue_duration_s":0.017034,"scheduling_latency_s":0.017065}2667gitlab # [ 393.510930] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:09.111Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"0afe284c171707b6a926ce8c","created_at":"2026-09-01T10:38:09.035Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8d5c4eb800c01f9714aa6c6742032b7","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:09.048Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-0afe284c171707b6a926ce8c: done: 0.045988 sec","job_status":"done","queue_duration_s":0.017034,"scheduling_latency_s":0.017065,"redis_calls":11,"redis_duration_s":0.00966,"redis_read_bytes":9,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00109,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005444,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003126,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016348,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000946769,"exclusive_lock_hold_duration_s":0.01411604,"duration_s":0.045988,"completed_at":"2026-09-01T10:38:09.111Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012123,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2668gitlab # [ 393.532295] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:09.114Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"7d1fa3521004de8364541688","created_at":"2026-09-01T10:38:09.067Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fcd4b0816d3b31307f2c442ce33e4e3c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:09.079Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-7d1fa3521004de8364541688: start","job_status":"start","queue_duration_s":0.03526,"scheduling_latency_s":0.035284}2669gitlab # [ 393.550921] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:09.143Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2670gitlab # [ 393.555351] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:09.155Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"7d1fa3521004de8364541688","created_at":"2026-09-01T10:38:09.067Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fcd4b0816d3b31307f2c442ce33e4e3c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:09.079Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-7d1fa3521004de8364541688: done: 0.0409 sec","job_status":"done","queue_duration_s":0.03526,"scheduling_latency_s":0.035284,"redis_calls":9,"redis_duration_s":0.006987,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000602,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00458,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001805,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013463,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0409,"completed_at":"2026-09-01T10:38:09.155Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011492,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2671gitlab # [ 393.610354] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:09.167Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"e0731a27f754d08622828b20","created_at":"2026-09-01T10:38:09.098Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f491a07d77585a24dee0414f606fb079","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-01T10:38:09.122Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-e0731a27f754d08622828b20: start","job_status":"start","queue_duration_s":0.04457,"scheduling_latency_s":0.044605}2672gitlab # [ 393.625066] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:09.203Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"e0731a27f754d08622828b20","created_at":"2026-09-01T10:38:09.098Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f491a07d77585a24dee0414f606fb079","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-01T10:38:09.122Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-e0731a27f754d08622828b20: done: 0.036529 sec","job_status":"done","queue_duration_s":0.04457,"scheduling_latency_s":0.044605,"redis_calls":9,"redis_duration_s":0.006277,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000903,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002438,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002936,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036529,"completed_at":"2026-09-01T10:38:09.203Z","load_balancing_strategy":"primary","db_duration_s":0.004133,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2673gitlab # [ 393.693906] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:09.210Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"9a2f0dc17161c27b2fb67c6c","created_at":"2026-09-01T10:38:09.136Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b20e7c6db09bdf782cd84ed73ffceb71","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:09.149Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-9a2f0dc17161c27b2fb67c6c: start","job_status":"start","queue_duration_s":0.06084,"scheduling_latency_s":0.060872}2674gitlab # [ 393.698305] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:09.256Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"9a2f0dc17161c27b2fb67c6c","created_at":"2026-09-01T10:38:09.136Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b20e7c6db09bdf782cd84ed73ffceb71","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:09.149Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-9a2f0dc17161c27b2fb67c6c: done: 0.045874 sec","job_status":"done","queue_duration_s":0.06084,"scheduling_latency_s":0.060872,"redis_calls":11,"redis_duration_s":0.010992,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001365,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005158,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004469,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016864,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.002313423,"exclusive_lock_hold_duration_s":0.003399594,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.045874,"completed_at":"2026-09-01T10:38:09.256Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003697,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2675gitlab # [ 393.765284] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:09.260Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"06442817bbed240fd22a5680","created_at":"2026-09-01T10:38:09.168Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c065ed9c639fac66f86d67a9bf083abb","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:09.177Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-06442817bbed240fd22a5680: start","job_status":"start","queue_duration_s":0.082911,"scheduling_latency_s":0.082934}2676gitlab # [ 393.867077] gitaly[932]: time="2026-09-01T10:38:09.467Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8REJXQ8GD00PXJG1VX82T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:38:19.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:38:09.462" grpc.time_ms=4.764 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2677gitlab # [ 393.938472] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:09 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 5492678gitlab # [ 393.939440] nixbot[3255]: {"ts": "2026-09-01T10:38:09.539Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e8d9360f6d71fb57cb350a711a26cf7895d79390 \"HTTP/1.1 201 Created\""}2679gitlab # [ 393.997940] gitaly[932]: time="2026-09-01T10:38:09.598Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8RF46V1M3XGSWNZRHCDNG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:39.597" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:09.597" grpc.time_ms=0.656 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2680gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses2681gitlab # [ 394.052838] gitaly[932]: time="2026-09-01T10:38:09.653Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1E8RF46V1M3XGSWNZRHCDNG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:38:19.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:38:09.647" grpc.time_ms=5.09 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2682gitlab # [ 394.093095] gitaly[932]: time="2026-09-01T10:38:09.693Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8RF7JQ9KV6KZSRQ1VZ75Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:39.692" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:09.692" grpc.time_ms=0.497 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2683gitlab # [ 394.106875] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:09 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1642684gitlab # [ 394.107990] nixbot[3255]: {"ts": "2026-09-01T10:38:09.708Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e8d9360f6d71fb57cb350a711a26cf7895d79390 \"HTTP/1.1 201 Created\""}2685gitlab # [ 394.134520] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:09 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses HTTP/1.1" 200 1313 "" "curl/8.21.0" 842686gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses, in 0.11 seconds)2687[{'id': 5, 'sha': 'e8d9360f6d71fb57cb350a711a26cf7895d79390', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 6s (1 warning)', 'created_at': '2026-09-01T10:38:02.137Z', 'started_at': None, 'finished_at': '2026-09-01T10:38:09.509Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'e8d9360f6d71fb57cb350a711a26cf7895d79390', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-09-01T10:38:09.682Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2688gitlab # [ 394.177048] gitaly[932]: time="2026-09-01T10:38:09.777Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8RFA0ZRXJHXVNSR3DKWMJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T10:38:39.776" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T10:38:09.776" grpc.time_ms=0.802 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2689gitlab # [ 394.217830] gitaly[932]: time="2026-09-01T10:38:09.818Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E8RFA0ZRXJHXVNSR3DKWMJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T10:38:19.173" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T10:38:09.813" grpc.time_ms=4.824 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2690gitlab # [ 394.263463] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:09 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e8d9360f6d71fb57cb350a711a26cf7895d79390 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1352691gitlab # [ 394.267762] nixbot[3255]: {"ts": "2026-09-01T10:38:09.868Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e8d9360f6d71fb57cb350a711a26cf7895d79390 \"HTTP/1.1 201 Created\""}2692gitlab # [ 394.393915] gitaly[932]: time="2026-09-01T10:38:09.994Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E8RFGYCVWAP4T2248NJWSA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:38:09.987" grpc.time_ms=6.486 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2693gitlab # [ 394.395590] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:09 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 452694gitlab # [ 394.425680] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:10.025Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"06442817bbed240fd22a5680","created_at":"2026-09-01T10:38:09.168Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c065ed9c639fac66f86d67a9bf083abb","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:09.177Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-06442817bbed240fd22a5680: done: 0.765295 sec","job_status":"done","queue_duration_s":0.082911,"scheduling_latency_s":0.082934,"redis_calls":564,"redis_duration_s":0.302664,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002231,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003017,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.297416,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.391566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.765295,"completed_at":"2026-09-01T10:38:10.025Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001457,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2695gitlab # [ 394.459677] gitaly[932]: time="2026-09-01T10:38:10.060Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E8RFJJSS104P5HSN9YR1S5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:38:10.053" pid=932 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2696gitlab # [ 394.460925] gitaly[932]: time="2026-09-01T10:38:10.060Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8RFJJSS104P5HSN9YR1S5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:38:10.053" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2697gitlab # [ 394.462534] gitaly[932]: time="2026-09-01T10:38:10.062Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E8RFJJSS104P5HSN9YR1S5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T10:38:10.053" grpc.time_ms=9.465 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=932 remote_ip="2001:db8:1::1" span.kind=server system=grpc2698gitlab # [ 394.519383] gitlab-workhorse[832]: time="2026-09-01T10:38:10Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672699gitlab # [ 394.520294] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/01:10:38:10 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 612700gitlab # [ 394.534446] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:10.134Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7598fa890efc6a6e045c4a24","created_at":"2026-09-01T10:38:09.184Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5a18af76ee274d71db4c47dd568ba3de","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:09.198Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7598fa890efc6a6e045c4a24: start","job_status":"start","queue_duration_s":0.935805,"scheduling_latency_s":0.935852}2701gitlab # [ 394.566065] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:10.166Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7598fa890efc6a6e045c4a24","created_at":"2026-09-01T10:38:09.184Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5a18af76ee274d71db4c47dd568ba3de","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:09.198Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7598fa890efc6a6e045c4a24: done: 0.031658 sec","job_status":"done","queue_duration_s":0.935805,"scheduling_latency_s":0.935852,"redis_calls":9,"redis_duration_s":0.002961,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000608,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001621,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000732,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031658,"completed_at":"2026-09-01T10:38:10.166Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00375,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2702gitlab # [ 394.567843] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:10.168Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"da96bdf55c815a69ab52de2f","created_at":"2026-09-01T10:38:09.207Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5e9e3cd712a3fb3fbdc4de68e9762a88","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:09.215Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-da96bdf55c815a69ab52de2f: start","job_status":"start","queue_duration_s":0.95228,"scheduling_latency_s":0.952315}2703gitlab # [ 394.773131] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:10.373Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"da96bdf55c815a69ab52de2f","created_at":"2026-09-01T10:38:09.207Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5e9e3cd712a3fb3fbdc4de68e9762a88","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:09.215Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-da96bdf55c815a69ab52de2f: done: 0.205155 sec","job_status":"done","queue_duration_s":0.95228,"scheduling_latency_s":0.952315,"redis_calls":13,"redis_duration_s":0.0032040000000000003,"redis_read_bytes":11,"redis_write_bytes":1567,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000912,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001134,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001158,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.157305,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000555378,"exclusive_lock_hold_duration_s":0.178395299,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.205155,"completed_at":"2026-09-01T10:38:10.373Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.052177,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2704gitlab # [ 394.829645] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:10.430Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"85c34317aba136eb077fc58a","created_at":"2026-09-01T10:38:09.227Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f303cd2fb05f47227d2f87240b4c6e1d","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:09.240Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-85c34317aba136eb077fc58a: start","job_status":"start","queue_duration_s":1.189805,"scheduling_latency_s":1.189835}2705gitlab # [ 394.875998] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:10.476Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"85c34317aba136eb077fc58a","created_at":"2026-09-01T10:38:09.227Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f303cd2fb05f47227d2f87240b4c6e1d","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:09.240Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-85c34317aba136eb077fc58a: done: 0.046341 sec","job_status":"done","queue_duration_s":1.189805,"scheduling_latency_s":1.189835,"redis_calls":12,"redis_duration_s":0.003002,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000444,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001366,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001192,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01838,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.007346185,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.046341,"completed_at":"2026-09-01T10:38:10.476Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003328,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2706gitlab # [ 394.877793] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:10.477Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"bc5f51dd19614b97ead783df","created_at":"2026-09-01T10:38:09.248Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"021343cd220385da7050a3057083e1fb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-01T10:38:09.259Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-bc5f51dd19614b97ead783df: start","job_status":"start","queue_duration_s":1.218408,"scheduling_latency_s":1.218433}2707gitlab # [ 394.955673] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:10.556Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2708gitlab # [ 394.962569] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:10.562Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"bc5f51dd19614b97ead783df","created_at":"2026-09-01T10:38:09.248Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"021343cd220385da7050a3057083e1fb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-01T10:38:09.259Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-bc5f51dd19614b97ead783df: done: 0.085123 sec","job_status":"done","queue_duration_s":1.218408,"scheduling_latency_s":1.218433,"redis_calls":9,"redis_duration_s":0.002049,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00049,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001139,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00042,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.066,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.066,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.085123,"completed_at":"2026-09-01T10:38:10.562Z","load_balancing_strategy":"primary","db_duration_s":0.066142,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2709gitlab # [ 394.963902] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:10.564Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"ff32e901daa064cf5aca257e","created_at":"2026-09-01T10:38:09.356Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5bd5955c71d0eaac820764946199cb10","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-01T10:38:09.373Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-ff32e901daa064cf5aca257e: start","job_status":"start","queue_duration_s":1.19116,"scheduling_latency_s":1.191184}2710gitlab # [ 394.975818] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:10.574Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2711gitlab # [ 394.981109] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:10.581Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"ff32e901daa064cf5aca257e","created_at":"2026-09-01T10:38:09.356Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5bd5955c71d0eaac820764946199cb10","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-01T10:38:09.373Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-ff32e901daa064cf5aca257e: done: 0.01718 sec","job_status":"done","queue_duration_s":1.19116,"scheduling_latency_s":1.191184,"redis_calls":9,"redis_duration_s":0.001867,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000388,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000995,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000484,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010111,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01718,"completed_at":"2026-09-01T10:38:10.581Z","load_balancing_strategy":"primary","db_duration_s":0.000519,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2712gitlab # [ 395.033173] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:10.582Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"d75b55197b0ddd03432b0260","created_at":"2026-09-01T10:38:09.398Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"84b85aa7c5e328138f7b06d1d5016140","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:09.407Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-d75b55197b0ddd03432b0260: start","job_status":"start","queue_duration_s":1.175524,"scheduling_latency_s":1.175547}2713gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses2714gitlab # [ 395.051932] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:10.606Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"d75b55197b0ddd03432b0260","created_at":"2026-09-01T10:38:09.398Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"84b85aa7c5e328138f7b06d1d5016140","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:09.407Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-d75b55197b0ddd03432b0260: done: 0.023478 sec","job_status":"done","queue_duration_s":1.175524,"scheduling_latency_s":1.175547,"redis_calls":9,"redis_duration_s":0.002161,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000493,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00127,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000398,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023478,"completed_at":"2026-09-01T10:38:10.606Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002962,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2715gitlab # [ 395.114544] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:10.607Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"e5987d041427425385091a21","created_at":"2026-09-01T10:38:09.416Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f929587720be64f9e94f867752c135ed","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:09.423Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-e5987d041427425385091a21: start","job_status":"start","queue_duration_s":1.18388,"scheduling_latency_s":1.183902}2716gitlab # [ 395.179141] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:10.627Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"e5987d041427425385091a21","created_at":"2026-09-01T10:38:09.416Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f929587720be64f9e94f867752c135ed","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:09.423Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-e5987d041427425385091a21: done: 0.019858 sec","job_status":"done","queue_duration_s":1.18388,"scheduling_latency_s":1.183902,"redis_calls":9,"redis_duration_s":0.0018570000000000001,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000377,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001087,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000393,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019858,"completed_at":"2026-09-01T10:38:10.627Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000433,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2717gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e8d9360f6d71fb57cb350a711a26cf7895d79390/statuses, in 0.13 seconds)2718[{'id': 5, 'sha': 'e8d9360f6d71fb57cb350a711a26cf7895d79390', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 6s (1 warning)', 'created_at': '2026-09-01T10:38:02.137Z', 'started_at': None, 'finished_at': '2026-09-01T10:38:09.509Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'e8d9360f6d71fb57cb350a711a26cf7895d79390', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-09-01T10:38:09.682Z', 'started_at': None, 'finished_at': '2026-09-01T10:38:09.839Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2719(finished: subtest: eval and build post commit statuses, in 8.86 seconds)2720(finished: run the VM test script, in 452.80 seconds)2721gitlab # [ 395.187237] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:10.629Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"39a25835bf89070fb13f6da9","created_at":"2026-09-01T10:38:09.435Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"75320983c3abea28a3a8f27fb8b4bf26","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-01T10:38:09.446Z","job_size_bytes":2,"pid":3276,"sidekiq_tid":"m8ig","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-39a25835bf89070fb13f6da9: start","job_status":"start","queue_duration_s":1.182584,"scheduling_latency_s":1.182613}2722gitlab # [ 395.260411] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:10.641Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2723test script finished in 452.82s2724cleanup2725kill QemuMachine (pid 46)2726gitlab # [ 395.261911] sidekiq-cluster[3276]: {"severity":"INFO","time":"2026-09-01T10:38:10.648Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"39a25835bf89070fb13f6da9","created_at":"2026-09-01T10:38:09.435Z","trace_propagation_headers":{"sentry-trace":"0cb1edab47cc42528a29a55ee4b668a7-2eb2813495d949ec","baggage":"sentry-trace_id=0cb1edab47cc42528a29a55ee4b668a7,sentry-environment=,sentry-release=Unknown"qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14)2727gitlab # }2728(finished: cleanup, in 1.76 seconds)